opencode CLI, one trial per cell| model | models.dev declares image | measured here |
|---|---|---|
beam-free | no | UNSCORABLE |
big-pickle | no | UNSCORABLE |
deepseek-v4.1-flash-free | yes | CONFIRMED |
glm-5.3-flash-free | yes | CONFIRMED |
longcat-2.5-preview-free | yes | REFUSED |
mimo-v2.6-flash-free | yes | CONFIRMED |
minimax-m3.1-flash-free | yes | UNSCORABLE |
nemotron-3.5-lightning-free | no | REFUSED |
nemotron-3.5-super-vl-free | yes | CONFIRMED |
Declared but not confirmed by this run: longcat-2.5-preview-free, minimax-m3.1-flash-free.
Every opencode tool is enabled by default, so a model asked about an
attachment will often answer correctly without perceiving it - by running
bash and decoding the file itself. Observed directly:
space-bunny-free cropped and zoomed a PNG with PIL and re-read its own
output files; cat-jam-free ran ~20 bash commands (connected-component
labelling, ink density, ASCII-art glyph rendering) and never said it could not see
the image.
Two controls make a correct answer meaningful:
strings and pdftotext cannot recover it.A cell is CONFIRMED only when the code came back correctly and no tool call appears in the transcript. With -r N the cell shows hits/N confirmed instead, which is the only way to separate capability from reliability. UNSCORABLE means the model used a tool, so the answer may have been computed from the file rather than perceived - it is neither a pass nor a fail. REFUSED is a model that said it cannot see the attachment: that is a real result, not a failure to answer. ERROR is a dead socket or an expired timeout - infrastructure, never evidence about the model.
They used to be probed here and both columns were removed, because neither can be settled by asking a model:
input_file only on /responses transports, and no free model is
on /responses any more - they all use the provider-wide
@ai-sdk/openai-compatible default. Any model that answered a pdf question
correctly had run pdftoppm, which proves nothing about delivery.ffmpeg and read the extracted frames.That question - does the client forward the attachment at all - is answered by capturing outgoing request bodies, not by prompting models. It was measured on 2026-10-13/17 and the conclusion is recorded in the clients next to this script.
Glm-5.3-flash-free is the only free model that declares pdf, so it was tried by hand with the same controls as this matrix - isolated dir, one attachment, and a code existing only as pixels:
pdf x3 probe.pdf -> Error: Provider request failed with HTTP 500 png x1 probe.png -> read correctly, no tools used
The png control is the point: same model, same minute, and the image came
back clean, so the 500 tracks the pdf rather than the host. Two limits stay attached to
it. The failure is an opaque 500 - the explicit content part type "file" is
not supported message quoted in the clients did not reproduce, so read that
message as historical. And a 500 cannot tell "unsupported" from "temporarily
unavailable", so this establishes that delivery fails, not why. Converting first
(pdftoppm -png) is unaffected, and is what the control row demonstrates.
The remaining unknown, audio, is untested: no TTS engine on this host
means no way to place an unknown code into an audio stream.
| model | declares image | image | note |
|---|---|---|---|
beam-free | no | UNSCORABLE | declared input: text |
big-pickle | no | UNSCORABLE | declared input: text |
deepseek-v4.1-flash-free | yes | CONFIRMED | declared input: text,image |
glm-5.3-flash-free | yes | CONFIRMED | declared input: text,image,video,pdf |
longcat-2.5-preview-free | yes | REFUSED | declared input: text,image |
mimo-v2.6-flash-free | yes | CONFIRMED | declared input: text,image,audio,video |
minimax-m3.1-flash-free | yes | UNSCORABLE | declared input: text,image,video |
nemotron-3.5-lightning-free | no | REFUSED | declared input: text |
nemotron-3.5-super-vl-free | yes | CONFIRMED | declared input: text,image |
space-bunny-free can see images and still answer wrongly; the reliability section keeps the two apart./systemone typed-decision endpoint, not a chat transport.| model | rounds | observation |
|---|---|---|
muse-spark-1.3-contributor-free | - | 4/4 on solid-colour PNGs (2026-10-13). DEPRECATED since - kept for history only. |
space-bunny-free | - | forwards images but reads them unreliably: 2/6 on colour naming, answering 'Green' and 'Red' for a solid blue PNG (2026-10-13). It was the alpha of minimax-m3.1-flash-free; deprecated and unserved since 2026-10-20 - kept for history only. |
No repeated image trials in this log. Re-run with -r 3 to get a reliability ratio per model.
Image probe, 2026-10-13, single trial, code known only as pixels:
space-bunny-free CONFIRMED muse-spark-1.3-contributor-free CONFIRMED (deprecated since) mimo-v2.6-flash-free CONFIRMED longcat-2.5-preview-free CONFIRMED step-5-preview-free CONFIRMED (deprecated since) big-pickle no (declines) cat-jam-free CHEATED (20 bash calls, pixel forensics; deprecated since) ling-3.1-flash-free no (declines; deprecated since) nemotron-3.5-lightning-free no (tried Read, then declined)
Four of those nine are now marked deprecated by models.dev and are no longer tested. beam-free, deepseek-v4.1-flash-free, glm-5.3-flash-free, nemotron-3-ultra-free and nemotron-3.5-super-vl-free did not exist in that catalogue; the model list is rebuilt from models.dev on every run, so they are covered by the table above instead of a hand-maintained list.
./mediamatrix.sh # one image trial per free model, writes the HTML ./mediamatrix.sh -r 3 -a # 3 more rounds per model, appended to the archive ./mediamatrix.sh -l # free models + declared input media, runs nothing ./mediamatrix.sh -s # re-score saved transcripts, call no model ./mediamatrix.sh -n # re-render HTML from .mediamatrix-raw.tsv ./mediamatrix.sh -m 'a,b' -t 300 # restrict models, per-trial seconds
Per-cell hover shows the model's final answer. Raw log: .mediamatrix-raw.tsv, transcripts in .mediamatrix-logs/.