Collecting documents from public sources and turning them into structured data — a job that cannot be done by hand.
A monorepo of a dozen tools and an admin panel: crawling URLs, downloading files, hash-based de-duplication, object storage, batch recognition by a language model and dispatch of the results. A scheduled job queue drives all of it.
The full write-up is still being written — this is the short version for now. Happy to walk you through the details.