vannevarlabs · Remote
Vannevar builds AI systems for the Department of War's most consequential missions. We have 125 deployments across every branch and combatant command spanning our core platform and five distinct products. An an example of how we work, when major combat operations started with Iran, we fielded a new product supporting 24/7 operations and 9,000+ users in three months. How we build is our advantage: we deploy forward with the people who own the mission. Our engineers, product team, and CTO deploy forward to the point of friction, including visiting units in Ukraine. We focus on embedding directly with operational units supporting great power competition with China, combat operations with Iran, and counter-narcotics missions. About the Role At Vannevar, we build mission software for the national security, defense, and intelligence communities. The Collection Platform team builds the systems that acquire mission-critical data from websites, APIs, feeds, and third-party platforms. In this role, you’ll be a hands-on technical leader for collection and ingestion systems. You’ll build web scrapers, third-party integrations, and data pipelines that produce complete, correct, and timely data despite unreliable upstream systems. You’ll partner across engineering, product, and mission teams to meet mission needs. What you’ll do Shape the architecture and technical direction of collection and ingestion systems, looking 6–12 months ahead to identify needs and develop plans to address them. Build and operate production web scrapers, third-party integrations, and data pipelines, delivering major services and infrastructure with impact across the organization. Design for schema changes, rate limits, authentication requirements, and partial failures. Set collection schedules based on source change frequency, user needs, downstream capacity, and cost. Build reliable retry, backfill, deduplication, idempotency, and replay mechanisms. Define and monitor data completeness, correctness, fr…
Sign in to see the original posting
Browse more developer jobs on RoleBurst