Backend Engineer, Streaming
Coactive
Coactive makes it easy to search, filter, and analyze visual content. Increasingly image and video data captures the content we watch, the products we buy, and the work we do, and already represents 80% of internet traffic. But rather than being an asset, visual content is often a tax or even a liability because it is so hard to work with and understand. Coactive solves this by bringing structure to unstructured visual data. Rather than spending months (or years) building complex infrastructure, data teams can unlock the value of their visual data in minutes to power use cases such as content understanding and moderation, search, and analytics.
Coactive was founded by experts who shaped the fields of high-performance deep learning and data-centric AI. We have the scars from building and working with the first generation of modern machine learning systems at Google, Meta, Pinterest, eBay, Lyft, and other leading organizations. Through our decades of experience, we have developed a playbook to democratize the toughest parts of machine learning systems; no PhD required.
This is an exceptional opportunity to work with one of Forbes' top 50 AI companies, backed by key investors such as Andreesen Horowitz and Bessemer Ventures. This position is truly a chance to be part of a category-defining team that is attracting top-tier talent interested in pushing the boundaries of what's possible in AI.
What we need:
We are looking for a Backend Engineer to develop and maintain scalable streaming data pipelines and workflows. Your role will involve integrating diverse data sources, transforming datasets for machine learning models, and ensuring efficient data processing. Collaborating with backend engineers, ML engineers, and product teams, you'll optimize pipeline performance and uphold data quality standards. You'll also provide expertise in best practices for ingestion frameworks and streaming pipeline development. Monitoring and observability implementation will be key to maintaining high data quality. Your work will shape our engineering practices and contribute to our growing company culture. We seek candidates with extensive experience in data engineering, distributed processing ecosystems like Spark and Kafka, and proficiency in programming languages like Python. Knowledge of database internals, query optimizations, and a strong testing ethos are essential.
What you’ll do:
- Design, develop and support resilient, scalable data pipelines and workflows that integrate various sources of structured and unstructured data
- Design and develop customizations and data wrangling needed to transform datasets in a way that is best suited for processing by our ML models.
- Collaborate closely with backend engineers, ML engineers and product teams to assess pipeline performance and improve results as needed.
- Provide guidance on best practices around ingestion frameworks and streaming pipelines development.
- Implement monitoring and observability to ensure data quality and efficient data processing.
- Shape engineering best practices and company culture as we grow
What you’ll do:
- A proven track record with 8+ years of professional experience in data engineering, building data pipelines, ETL, streaming data ingestion, distributed processing and databases with large datasets.
- Experience in distributed processing ecosystems like Spark, Flink, Kafka, Kubernetes, etc.
- Experience designing systems with event driven architecture/streams and message queues.
- Proficiency in design of algorithms, data structures, design patterns and deploying scalable microservices.
- Deep knowledge of database internals, query planning and optimizations
- Experience with one or more general purpose programming languages including but not limited to Python, Go or Java (we use Python)
- Experience working with NoSQL databases including but not limited to MongoDB, ElasticSearch, etc and relational databases like Postgres, etc.
- Strong testing ethos and knowledge of best practices, including code quality, accessibility, performance, and security
What you can expect from us:
- Location: San Jose, California (hybrid, with three days in office)
The estimated annual base salary for this position is between $180,000-$220,000.*
At Coactive, cash salary is only one part of our total compensation package. Other benefits for this position include, but are not limited to:
- Market leading equity grants
- 100% medical, dental, & vision coverage for you
- Medical, dental, & vision partially covered for your dependents
- Unlimited PTO
- Social events ranging from book clubs, happy hours, and hiking to board game nights and games of Mario Kart.
Further, you can expect a supportive work environment from us. We build products, but we develop people.
*Actual pay is dependent on an individual candidate’s professional background, experience, skills and qualifications, as well as market demand and business demands. This pay range is subject to change and may be modified in the future. The salary, other compensation, and benefits information is accurate as of the date of this posting.
We embrace and celebrate the diversity of our employees. We are committed to equal employment opportunity regardless of race, color, ethnicity, ancestry, religion, national origin, gender, sex, gender identity or expression, sexual orientation, age, citizenship, marital or parental status, disability, veteran status, or other class protected by applicable law. We are proud to be an equal opportunity workplace.
We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please contact us to request accommodation.