DFYCollect · For the technical evaluator
How DFYCollect finds every document, and proves it.
What it does with every document, what it measures on real data, and what it can't do yet, for whoever has to check our claims.
One pass
How a run works
Each document is found, identified, opened, searched and, if it matches, copied out as the run reaches it. Nothing is gathered first and searched later.
Identify
What each file really is, not just what its name says.
Open
Mail stores, archives and attachments, to any depth.
Read
Metadata and text, and the text is kept.
Search
Against your terms and dates, as each document is read.
Copy
Matches, with their families.
Same engine either way
The window and the command line run the same engine, so a job scheduled overnight behaves exactly like one started by hand.
Design decisions
What that means in practice
Each of these is a choice made so that a collection can be explained afterwards, not just described.
Size is limited by disk, not memory
Each document is searched on its own as it is read, so a search never has to hold the whole collection in memory.
The text is kept anyway
Extraction is where the time goes, and it happens whether a document matches or not. Keeping the text costs disk rather than time, and turns a new set of terms from a multi-hour re-run into a query.
Skipped only when it provably can't match
With a date range set, a file is ruled out unopened only when its dates prove it. A file dated after the range is still opened, because a message copied out of a mailbox today may have been sent years ago.
Identified by what it is
In a mail export, a file named mime001.txt is routinely a spreadsheet. When
a name can't be trusted, DFYCollect checks what the file actually is.
Unparseable text is salvaged, and says so
Damaged files and formats with no reader still give up their readable runs — one legacy document yielded 162,000 usable characters. Every one carries a warning, so a reviewer can tell we read this from we scraped what we could.
Families are ordered before anything is numbered
Numbering first would scatter attachments away from the messages they belong to, which is the one thing a load file must never do.
Every word is searched
There are no ignored words: the, of and not are searched like
any other. For the cases where punctuation matters, exact:"…" matches the
characters themselves.
Names inside paths are searchable
Searching casefile finds casefile.pst, and a custodian's name is
found inside a server path.
End to end
A whole run, which is a different number
The mail-store figures measure one thing: messages read out of a store. A whole run also extracts, hashes, indexes and copies, so its rate is an order of magnitude lower. The two are easy to mistake for a contradiction.
One run, twelve workers
3 h 35 m
for 1,193,546 documents, 92.4 a second
768,979 messages, 314,678 attachments, 7,554 archive entries and 634 containers, in September 2026.
Text kept
1,084,350
documents with their text stored
So searching the whole collection again for a new term takes minutes rather than another three and a half hours.
Production
2 h 27 m
to produce the 31,233-document set
A further step on top of the run, numbering families together and writing the set out.
Scans, photos and faxes
Text recovery
Reading scans is a second pass over the collection, and on an image-heavy corpus the longest step. Four things decide how long it takes, and what it finds.
A long scan doesn't hold up the rest
Pages from different documents are read side by side. On twenty test pages a pass went from 97.5 seconds to 53.5, with the text identical, character for character. A page takes about a second when it's sparse and about ten when it's dense.
Identical documents are read once
Documents are grouped by hash before anything is read; one of each group is read and the rest take its text, recorded as having done so. On the collection on the DFYCollect page, 42,853 documents meant 9,067 reads.
Other machines can read pages too
A helper holds none of the collection: it receives pages and sends back the words over an encrypted connection, and only helpers you've listed are trusted. If one drops out, its pages are read on the main machine and no document fails.
Or straight after the run
Text recovery can run as soon as the collection finishes, unattended. When a production follows, what it reads is checked against the terms first, so a scan whose only mention of a term was in its picture is produced.
What it reads
Coverage
And what it does with the files it can't read: records them, counts them, and moves on.
PST and OST stores, loose .msg and .eml, attachments to any
depth. A mailbox Outlook has open is read from a shadow copy of the disk, which needs
administrator rights.
Archives
Containerszip, 7z, rar and tar, with size, output and depth ceilings against zip bombs.
Documents
Office and PDFWord, Excel, PowerPoint, PDF, Visio, Project and OneNote.
Embedded
Inside documentsObjects inside Office documents, and pictures pasted into them, above a size floor.
Charts
LegacyCharts, org charts and equations from programs nothing opens any more, read into the document holding them, with every figure and name searchable.
Scans
ScansRead on request or straight after the run, at a chosen resolution, including PDFs where only some pages are images.
Unparseable
SalvageText salvaged from damaged files and formats with no reader, with a warning on each.
Resilience
Nothing ends a jobUnreadable folders, locked files, corrupt stores and password-protected documents are recorded, counted and stepped over. An interrupted run resumes; a cancelled one keeps what it had written.
Where it runs
On your own computers
No document is sent to an outside service.
Nothing to install
A Windows desktop application and a command line, which run from a folder copied onto the machine.
Ready for review anywhere
A production loads into almost any review platform. Prepared for DFYReview, it also carries where each document was found, its dates with their time zones, and how its text was read.
Current limits, stated plainly
The section most vendors leave out. If any of these matter to your matter, they're worth knowing before a trial rather than during one.
New terms, original run
SearchChanging the terms of the original collection means running it again. Searching the finished collection again does not; that's what the kept text is for.
Scans have no text in the run
ScansText recovery reads them afterwards. Run straight after the collection, a scan that matches is produced; started later by hand, it's found by searching again.
Reading scans is slow
ScansAbout a second a page for a sparse scan and ten for a dense one on one machine. Helper machines add their own processors, but on an image-heavy corpus it's still the longest step.
A stopped document starts over
ScansIf text recovery is stopped part-way through a document, that document is read again from its first page.
One worker an archive
SpeedArchives are processed by a single worker; only mail stores are split across several.
A mail store resumes whole
ResumeA run resumes file by top-level file, so an interrupted mail store starts again from its beginning.
No EXIF, no transcription
MediaImages, audio and video carry path and file-system metadata only. DFYAI can transcribe the audio and video when it loads the production, so what is said becomes searchable there.
PowerPoint 95
FormatsPresentations saved purely in the PowerPoint 95 format can't be read. Files saved for both 95 and 97 are read from their 97 half, and say so.
WordPerfect and TNEF
FormatsIdentified, but not parsed as their structure; their text is salvaged.
Evaluating DFYCollect for your firm?
We'll walk your technical team through it
In as much detail as they want, with the guides and against the kind of data your matters hold.