The Batch Attachment Uploader currently matches attachment files to records by parsing the filename itself against a selected matching field (e.g. Catalog Number). This works well when the field has a UIFormatter with no regex parts, since Specify can reliably derive the field value from the filename (and normalize things like leading zeros).
It breaks down for institutions where the matching field is free text or contains non-numeric characters — e.g. UMMZI-90126 at UMich. In these cases:
- Single-attachment-per-record uploads mostly work, because the whole filename (minus extension) is compared to the field value.
- Multi-attachment-per-record uploads do not work, because the uploader's
_suffix convention (UMMZI-90126_dorsal.jpg) requires a strict, regex-free format to reliably split "the part that identifies the record" from "the part that's just a disambiguating suffix." Free-text fields don't have this structure, so the tool can't tell where the catalog number ends and the suffix begins.
Specify 6 solved this with an attachment mapping file, which was an explicit CSV that pairs each filename with the record identifier it belongs to, so the record/filename association never has to be inferred from the filename's structure. This decouples matching from field formatting entirely, and naturally supports many-files-per-record.
Goals
- Let users provide an explicit filename → matching-field-value mapping, so multi-attachment matching works regardless of the matching field's format (or lack of one).
- Build this as a second mode within the existing Batch Attachment Uploader, using the same validation, disambiguation, upload, and rollback logic.
- Support many attachments per record via the mapping file, with no constraint on filename structure.
Non-Goals
- Not replacing filename-pattern matching. That remains the default/existing mode for institutions whose formatted fields already work well with it.
- Not auto-generating mapping files from existing data (out of scope for v1).
- Not supporting arbitrary non-tabular mapping formats, CSV only.
Proposed Solution
Add a Matching Mode choice when creating a new attachment dataset:
- Match by filename (current/default behavior) — unchanged.
- Match by mapping file (new) — user uploads a CSV that explicitly lists which record each file belongs to, instead of relying on the filename's format.
Workflow for mapping-file mode
- User selects the matching field, same picklist as today (Catalog Number, GUID, etc.) so this is what determines which record a row matches to.
- User selects "Match by mapping file" as the matching mode.
- User uploads a CSV. Specify presents a column-mapping step (similar to WorkBench import) so the user indicates which CSV column is the match value and which is the filename. (See open question on a remarks column.)
- User selects/drags in the actual image files, same as today.
- Specify cross-references the CSV against the selected files by exact filename match without parsing or normalization of the filename needed, since the record association comes from the CSV row, not the filename's structure.
- Record matching (CSV match value → database record) reuses the existing disambiguation logic: single match, no match, or multiple matches, exactly as filename mode does today.
Example (from the UMich data)
| Catalog Number |
Attachment Name |
| UMMZI-90126 |
UMMZI-90126.jpg |
| UMMZI-90175 |
UMMZI-90175.jpg |
| UMMZI-90175 |
UMMZI-90175_1.jpg |
| UMMZI-90176 |
UMMZI-90176.jpg |
| UMMZI-90176 |
UMMZI-90176_1.jpg |
Two rows sharing a catalog number simply means two attachments go to the same record — no special suffix convention needed, since the pairing is explicit.
UI / Data Model Changes
- Dataset creation dialog: add a "Matching Mode" selector (Filename Pattern / Mapping File).
- CSV upload + column picker: reuse WorkBench-style mapping UI for choosing which columns are "Match Value" and "File Name."
- Dataset table: in mapping-file mode, the field-value and filename columns are populated from the CSV rather than derived from the filename; "Incorrectly Formatted" no longer applies in this mode (there's no format to violate).
- Dataset metadata: store the matching mode and column mapping on the dataset record so it's preserved on reload/resume, consistent with how the existing filename-mode config is stored.
Edge Cases & Validation Rules
- CSV row references a filename that wasn't uploaded → flag as
File Missing.
- An uploaded file isn't referenced by any CSV row → flag as
Not in Mapping File.
- Duplicate filename entries within the CSV → validation error before matching proceeds.
- CSV match value has no matching record, or matches multiple records → reuse existing
No match / Multiple Matches. Click to Disambiguate behavior.
- Filename case sensitivity — should be exact match, but worth surfacing a warning if a case-only mismatch is detected (common when files move between OS's).
- Large CSVs — should be validated/matched at the same scale the tool already handles for filename mode.
- Rollback behaves identically in mapping-file mode.
Discussed in #5941
Originally posted by grantfitzsimmons April 9, 2024
Is your feature request related to a problem? Please describe.
Yes, the catalog number field is a text field in several of the UMich databases. This enables support for the non-number characters they store in the field, like the UMMZI- suffix in this case.
In S6, we have been using an attachment mapping file which describes the catalog numbers and attachment filenames explicitly. I'm attaching an example here for reference. If a similar system could be added to S7, that would be great.
In this example, this mapping file is necessary to work around an issue with the catalog number field being formatted.
See this video to see how it worked: https://discourse.specifysoftware.org/t/attachment-mapping-file-video/101?u=grant
From @realVinayak:
Yes, unless the catalog number field itself is formatted (has a formatter other than text), then multiple records would not be matched. Specify 6 "got around" the issue via the mapping file
Here's an example file sent to us by Garth at UMich:
Example mapping file:
| Catalog Number |
Attachment Name |
| UMMZI-90126 |
UMMZI-90126.jpg |
| UMMZI-90159 |
UMMZI-90159.jpg |
| UMMZI-90171 |
UMMZI-90171.jpg |
| UMMZI-90175 |
UMMZI-90175_1.jpg |
| UMMZI-90175 |
UMMZI-90175.jpg |
| UMMZI-90176 |
UMMZI-90176_1.jpg |
| UMMZI-90176 |
UMMZI-90176.jpg |
| UMMZI-90179 |
UMMZI-90179_1.jpg |
| UMMZI-90179 |
UMMZI-90179.jpg |
| UMMZI-90209 |
UMMZI-90209_1.jpg |
| UMMZI-90209 |
UMMZI-90209.jpg |
| UMMZI-90231 |
UMMZI-90231_1.jpg |
| UMMZI-90231 |
UMMZI-90231.jpg |
| UMMZI-90234 |
UMMZI-90234_1.jpg |
| UMMZI-90234 |
UMMZI-90234.jpg |
| UMMZI-90237 |
UMMZI-90237.jpg |
| UMMZI-119907 |
UMMZI-119907_1.jpg |
| UMMZI-119907 |
UMMZI-119907.jpg |
| UMMZI-119908 |
UMMZI-119908_1.jpg |
| UMMZI-119908 |
UMMZI-119908.jpg |
Describe the solution you'd like
It should be implemented as it was in Specify 6.
Describe alternatives you've considered
They tried to import using the batch attachment uploader but could not indicate multiple images associated with a single specimen due to the formatting for that field.
Reported By
Garth at University of Michigan
The Batch Attachment Uploader currently matches attachment files to records by parsing the filename itself against a selected matching field (e.g. Catalog Number). This works well when the field has a UIFormatter with no regex parts, since Specify can reliably derive the field value from the filename (and normalize things like leading zeros).
It breaks down for institutions where the matching field is free text or contains non-numeric characters — e.g. UMMZI-90126 at UMich. In these cases:
_suffixconvention (UMMZI-90126_dorsal.jpg) requires a strict, regex-free format to reliably split "the part that identifies the record" from "the part that's just a disambiguating suffix." Free-text fields don't have this structure, so the tool can't tell where the catalog number ends and the suffix begins.Specify 6 solved this with an attachment mapping file, which was an explicit CSV that pairs each filename with the record identifier it belongs to, so the record/filename association never has to be inferred from the filename's structure. This decouples matching from field formatting entirely, and naturally supports many-files-per-record.
Goals
Non-Goals
Proposed Solution
Add a Matching Mode choice when creating a new attachment dataset:
Workflow for mapping-file mode
Example (from the UMich data)
Two rows sharing a catalog number simply means two attachments go to the same record — no special suffix convention needed, since the pairing is explicit.
UI / Data Model Changes
Edge Cases & Validation Rules
File Missing.Not in Mapping File.No match/Multiple Matches. Click to Disambiguatebehavior.Discussed in #5941
Originally posted by grantfitzsimmons April 9, 2024
Is your feature request related to a problem? Please describe.
In this example, this mapping file is necessary to work around an issue with the catalog number field being formatted.
See this video to see how it worked: https://discourse.specifysoftware.org/t/attachment-mapping-file-video/101?u=grant
From @realVinayak:
Here's an example file sent to us by Garth at UMich:
Example mapping file:
Describe the solution you'd like
It should be implemented as it was in Specify 6.
Describe alternatives you've considered
They tried to import using the batch attachment uploader but could not indicate multiple images associated with a single specimen due to the formatting for that field.
Reported By
Garth at University of Michigan