Cognitive Collective

Helping you find your next career in AI. Learn more about the job board on the Scale blog.

Are you a scaling AI startup? Email maxwell@scalevp.com to be added to our board.
131
companies
2,425
Jobs

Technical Program Manager, Human Data

OpenAI

OpenAI

IT, Operations
San Francisco, CA, USA
Posted on Thursday, March 21, 2024

About the Team

The Human Data team is focused on working with AI teachers to train OpenAI’s models to solve complex problems in order to provide value to humanity. We believe that highly skilled AI teachers are the key to unlocking the potential of our models to be as accurate and helpful as the experts themselves.

This role directly contributes to groundbreaking research, improving our multi-modal flagship models and products such as ChatGPT, GPT4o and Sora, and our AGI safety initiatives such as those executed by the Preparedness, and Safety Systems teams.


About the Role

The Human Data team works with a community of expert AI trainers and students which support OpenAI research. In this role, you will own the outcomes related to recruiting expert AI trainers, defining and delivering tailored value propositions that keep the expert trainers engaged and contributing high quality data, and building out the operational infrastructure to onboard expert AI trainers onto a broad range of OpenAI research projects such as data collection projects, model evaluations.

In this role, you will build a sustainable pipeline of expert AI trainers through a multitude of creative initiatives including but not limited to, forging university partnerships and student recruiting initiatives, focused prospecting for highly specific AI trainer profiles, and through the development of OpenAI Forum community programming and member cultivation which fuels the flywheel of expert AI trainer referrals.

The TPM Expert Trainer Program will also define the operational systems that connect the expert trainers to the OpenAI research projects, and own the outcomes of related data collection campaigns. This role requires a unique combination of skills related to prospecting, growth, and community building, as well as an expert level ability to define and lead ML data collection operations.

Collectively as a team, we are writing the playbook for how data collection efforts are accomplished with utmost integrity, not just following one. Members of this team are expected to develop the systems, processes, and tools to run data collection efforts in a way that takes advantage of OpenAI’s incredible product capabilities and unique research insights.

This role is based in San Francisco, CA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.

In this role, you will:

  • Own the outcomes of our data collection campaigns requiring expert AI trainers, including

    • Collaborating with the researcher(s) to understand their needs, find creative solutions to their data collection challenges

    • Recruiting AI trainers, and building long term, sustainable pipelines expert AI trainers leveraging the OpenAI Forum community, university partnerships, and other creative initiatives

    • Refine and scale project management systems related to AI trainer recruitment

    • Design and execute the onboarding process for campaigns

    • Ensuring our AI trainers have the materials and insights needed for success

    • Defining and owning communication feedback loops between researchers, ops and AI trainers

    • Own metrics of success related to expert AI trainer recruitment and retention

    • Own quality and throughput outcomes related to data collection campaigns utilizing expert AI trainers

  • Own the expert AI trainers database, maintain data hygiene, continue to refine search functionality so that the broader Human Data operations team can easily search for and find lists of expert trainers available to participate in data collection campaigns.

  • Partner with our vendors, OpenAI researchers, Human Data engineers to create a world class experience for all AI trainers working on OpenAI projects through implementation of educational materials and engagement opportunities (researcher office hours, working info sessions, OpenAI Forum community events for AI trainers)

  • Build and enhance the systems we’ll need as a team, including

    • Designing the overall training and onboarding strategy and materials for our researchers and ops team in relationship to expert AI trainers program

    • Creating dashboards, command centers, and queries that highlight campaign-specific and team-wide progress on recruiting, AI trainer skill sets, availability of trainers, AI trainer activities (which projects are they currently on, which have they previously worked on, at what quality?)

    • Building the frameworks, tools, and processes to understand our data, using SQL, Python, or other technical means when appropriate

  • Collaborate with the manager of the OpenAI Forum community to maintain accurate expert AI trainer membership database; plan and execute engagement opportunities for the expert AI trainers in the ForumPlan and execute partnership programming and community programming

You might thrive in this role if you:

  • You are an expert in project management. You are skilled at gathering and defining requirements, outlining and assigning clear roles and responsibilities, inspiring buy-in from cross functional partners, and can manage multiple., complex, high priority projects at once. PMP preferred.

  • Managed operations for data collection campaigns with humans in the loop: define the perfect AI trainer profile given the project goals and requirements, understand how to define and measure quality, have deep understanding of how to incentivize quality and throughput.

  • Excellent written and verbal communicator. You know how to discern the appropriate voice to address a distinct audience and can skillfully tailor the tenor and tone of your writing to engage different stakeholders; you are clear and concise.

  • Have experience building community and identifying appropriate value propositions that keep members engaged and contributing. You enjoy programming a space to activate participants. (office hours, info/educational sessions, roundtable discussions, expert talks)

  • You have experience building partnership programs (universities, student groups, community orgs)

  • A background in fundraising (major donor cultivation, events), University recruiting, and/or growth marketing, are all highly relevant skills to this position.

  • Working cross functionally brings you joy. You are equally capable and comfortable collaborating with software engineers and ML researchers as you are academics, lawyers, artists, undergraduate students, and more.

  • Ambiguity is your canvas and you thrive when given the agency to create something from nothing.

  • You have a strong desire to drive impact; your work is scalable and reproducible across the organization and external stakeholders find collaboration with you smooth and low lift.

About OpenAI

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.

We are an equal opportunity employer and do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, veteran status, disability or any other legally protected status.

For US Based Candidates: Pursuant to the San Francisco Fair Chance Ordinance, we will consider qualified applicants with arrest and conviction records.

We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.

OpenAI Global Applicant Privacy Policy

At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.