Cognitive Collective

Helping you find your next career in AI. Learn more about the job board on the Scale blog.

Are you a scaling AI startup? Email to be added to our board.

QA Analyst



IT, Quality Assurance
Kitchener, ON, Canada
Posted on Thursday, April 18, 2024

About Dialpad

Work Beautifully

Dialpad is the leading Ai-Powered Customer Intelligence Platform that is transforming how the world works together. Based on 4 billion minutes of analyzed voice and meetings data and growing, we have designed one, beautiful workspace that seamlessly combines the most advanced Ai Contact Center, Ai Sales, Ai Voice, and Ai Meetings with Ai Messaging. More than 30,000 innovative brands and millions of people use Dialpad to unlock productivity, collaboration, and customer satisfaction with real-time Ai insights. With initial funding and leadership from Google and leading venture capitalists such as ICONIQ and Andreessen Horowitz, Dialpad has over $200M in ARR and is one of the fastest growing Ai companies in the world.

About the team

Dialpad’s QA Analysts work closely within our AI teams to ensure that our customers can reap the benefits of AI without a high risk of unexpected outcomes or problematic outputs. In this role, you’ll combine test set creation and curation with automated, performance, and manual testing to ensure the best possible performance, security, and user experience for our AI features.

Dialpad’s AI Data team plays an essential role in ensuring that we are handling our customer data with care, creating representative datasets for testing, and verifying the outcomes of models are what we expect in output and performance. This team collaborates closely with the automatic speech recognition, natural language processing, data analysis, and AI Engineering teams to constantly improve our AI models as well as create new models that drive all of Dialpad’s AI features.

Your role

As a QA Analyst, you’ll own testing all of our new AI models from speech recognition to various ML and LLM models. You’ll work closely with our data annotation team, AI scientists and engineers to determine what tests need to be run, create test plans, design and create test sets, and execute on those test plans for new features or model architectures. You’ll also help run existing tests on our regular cadence releases that ensure we have no serious regressions while keeping our models up to date for our customers.

This position reports to our Manager, Data Annotation and Testing and has the opportunity to be based in our Kitchener office with the ability to work remotely in the area.

What you’ll do

  • Given an AI feature, be able to determine what tests should be run and what data will be required to ensure quality.
  • Create test plans and scripts for all levels of testing from smoke tests to unit tests to performance and security and report issues in JIRA.
  • Perform automated and manual tests of new and existing features to ensure they are functioning as expected.
  • Execute and coordinate red teaming, smoke testing, system integration testing, regression testing, security testing, performance testing, and end-to-end testing in support of test plans.
  • Create and design tests and metrics to measure both qualitative and quantitative aspects of ML models.
  • Be able to communicate with both technical and non-technical audiences to report the results of your testing.
  • Able to collaborate with cross-functional teams(ASR, NLP, Analytics) to align on testing objectives and requirements.

Skills you’ll bring

  • 3+ years of experience as a QA Analyst or equivalent role testing ML models.
  • Experience with Python, Docker, SQL, Google Cloud and Data Visualization tools.
  • Familiarity with version control tools like Git for collaborative projects.
  • Knowledge about machine learning models (ASR and NLP).
  • Experience with automated, performance, and manual testing.
  • Flexible with regard to changing priorities & business needs.
  • Strong analytical and problem-solving skills.
  • Ability to quickly learn and attention to details.
  • Excellent written and verbal communication skills.
  • If you speak or read other natural languages (e.g., Spanish, French) that would be an added bonus.

Dialpad benefits and perks

Benefits, time-off, and wellness

An apple a day keeps the doctor away—and it doesn’t hurt that we offer flexible time off and great options for medical, dental, and vision plans for all employees. Along with that, employees also receive a monthly stipend to help cover your cell phone bill, home internet bill, and we reimburse for gym membership costs, a variety of wellness events, and more!

Professional Development

Dialpad offers reimbursement for expenses related to professional development, up to an annual limit per calendar year.

We’ve been named a Top Workplace seven times, and a big part of this is because of our collaborative culture that elevates our teammates, celebrates wins, and brings together passion and talent.

Compensation and equity
Teamwork makes the dream work, and Dialpad offers competitive salaries in addition to stock options because each and every Dialer participates in our success.

Diversity, Equity, and Inclusion (DEI) at Dialpad

At Dialpad, we are passionate about Doing the Right Thing. This means we are committed to building a values-driven culture that celebrates identity, inclusion and belonging. As a global company, it’s our responsibility to come together to create a culture where all Dialers can Work Beautifully, Delight Our Users, and Innovate Continuously to bring our world-class product to life.

Every Voice Matters at Dialpad. We build community through our Employee Resource Groups, company-wide celebrations, service days, and a robust internal learning & development program focused on the success of our Dialers.

Don’t meet every single requirement? Studies have shown that women and marginalized groups are less likely to apply to jobs unless they meet every single qualification. At Dialpad we are dedicated to building an inclusive and authentic workplace, so if you’re excited about this role but your past experience doesn’t align perfectly with every qualification in the job description, we encourage you to apply anyways. You may be just the right candidate for this or other roles.

Dialpad is an equal-opportunity employer. We are dedicated to creating a community of inclusion and an environment free from discrimination or harassment.