Ava recrute un(e)
MLOps Engineer
CDI • Royaume-Uni
Cette offre est lié au département 2 - ai
Help us build out AI systems that help millions!
Many companies are building AI products these days, but few have a mission worth getting behind.
We’re working hard to empower 450 million deaf and hard-of-hearing people in their everyday life(e)Working at Ava means making an actual difference to people every day(e)Our users need our product to be the best it can be every day.
That’s why we need you to help us build and maintain the best possible AI deployment systems, to make sure that we deliver what our users rely on, all day, every day(e)Our product integrates cutting-edge research in diarization as well as humans-in-the-loop, making for a diverse and challenging but also very rewarding set of tasks.
All this needs to be deployed without breaking the bank, and needs constant monitoring as we update the different models involved(e)If you take pride in building efficient, maintainable ML/AI deployment pipelines that deliver low-latency inference at minimal cost, and if you have a mindset of “measure first” when it comes to AI model iterations, then we’d love to hear from you!
Read the full description here: https://www.notion.so/ava/48f740c4208d4f3493ff7df09c04844e
The tech stack we use at Ava is not a secret(e)In fact, most of it is fairly standard, it’s more what we do with it than what tools we use to do the job that makes us special(e)Feel free to reach out to us to ask questions about what we use and how; we want you to know exactly what you’re getting into(e)Talk to our engineers, they’re a cool bunch! 🙂
To be clear: this is a description of where we are, not where we want to be, or where we’ll stay(e)If you immediately start thinking of obvious ways to make this better: great, let’s compare notes!
So without further ado, here’s how our service currently runs, in very broad outline:
Our Stack
- REST & Websocket API endpoints in NodeJS express, overwhelmingly TypeScript(e)We still have a few any casts in there(e)They’ve been becoming rare though(e)😄
- Further-back backend (so..(e)the not client-facing one) also in Node/TS that joins together the multiple audio streams for each conversation.
- Scientific backend (this is where the ML/DL happens) in python with both tensorflow and pytorch elements(e)→ This is what a lot of your work would center around, at least in the beginning.
- 👆🏽These three components pass messages via Redis, the whole thing is pretty much event-driven(e)You have been warned(e)🙂
- Storage in Firebase, MongoDB, S3
- Logging and monitoring in Elastic/Kibana
- Infrastructure hosted on AWS, with use of ECS, EB, SQS, mostly managed with Terraform.
- Jenkins for CI/CD
What you'll be doing
- Lead the way on our AI deployment infrastructure strategy. Models are often built with quality of results and inference speed in mind(e)Deploying and running inference at scale however comes with its own set of slightly different challenges like inference throughput and cost(e)In our setting, stateful inference has to happen in real-time and with good (horizontal) scaling, robustly and allowing for fast fail-over when things do go wrong(e)To be clear here, we’re looking for someone who knows things we don’t know (yet) so this role comes with the responsibility, but also the freedom to make a lot of technical choices yourself.
- Weave metrics and monitoring into the fabric of the company(e)There are many places and times when we want to evaluate the quality of different models and processes: during development, on model changes, regularly to keep tabs on external API providers, and continuously in production to monitor the performance of our human/AI collaboration(e)You get to shape where and how to run these metrics, make sure that they’re available to the people who can see them, and presented in a clear and actionable way.
- Build logging & monitoring components that allow us to track specific use cases, failure scenarios, and the quality of the models’ results, keeping their performance from slipping over time(e)Where did that 3% increase in word error rate come from? We want to catch these things before the model goes out to the customer.
- Package up trained models in a way that is maintainable and doesn’t break the bank(e)It’s great to find an openly available model that does just what you need, but that doesn’t mean we should deploy it in all its float64 glory(e)Quantizing & pruning will still be necessary and even if you don’t do this yourself, then you’ll be working with the people who do, to make sure they have the tools to understand the runtime & quality impact of the changes being made.
What we need you to bring to the table
- You master Python(e)The core language & standard library should hold no secrets to you(e)Your python code is idiomatic and efficient(e)Experience with the common AI libraries (pytorch, tensorflow, scikit-learn etc) is a big plus.
- Deploying applications to AWS and working with the various AWS services is second nature to you(e)Learning about new AWS services is natural to you(e)Docker is table stakes.
- You understand, in detail, the trade-offs involved in different deployment options(e)When do we use a microservice, and when is that not the right decision?
- You’re passionate about technology; you don’t only want to get the job done, you also want the result to be maintainable and performant in the future.
- Your English is good enough to communicate efficiently in spoken and written form(e)French is bonus(e)(remember: everyone has an accent, that’s not the point here)
Bonus round: where you can shine
- You’re comfortable in NodeJs and TypeScript(e)Sure, most of your work will happen in other parts of the backend, but this will help you in understanding and communicating better with your colleagues.
- You have multiple years of experience building ML/AI systems that scale(e)You know the common tools that AI teams work with and what the benefits & drawbacks of these are.
- Building, modifying, and training models in pytorch, tensorflow, and (extrabonus) pyannote.
- You know what torch.jit does and have integrated ONNX into python and nodejs applications.
- Experience building systems with (soft) real-time constraints.
- Experience working with streaming audio data and/or speech data.
- Good understanding of Deep Learning / Machine Learning algorithms(e)You’ll be working with our research team; shared vocabulary will be a plus.
- Knowledge of modern monitoring and logging platforms: ELK, but also more machine learning specific monitoring of non-trivial metrics
What's special about this role?
- We’re a mission driven company. Everyone is here because we want to make a real impact(e)And impact we make(e)You’ll be changing lives with what you do.
- Work on and with cutting edge research results. As a real team-player you’ll quickly see that you’re enabling a bunch of really clever people with your work(e)And you’ll quickly see that your work is truly appreciated.
- Your expertise will be invaluable to make the right technical decisions. You’re joining in a moment where we know what we want to build, but there are many options for how to build it(e)Also we’re still small enough that your mere presence and attitude will shape the culture of the company in your image.
- Work at the cutting edge of a new engineering field. MLOps is new and super fast-moving(e)New tech is being developed at break-neck speed(e)And you’ll be at the center of this, picking the right tools and diving deep into them.
- Work under interesting and challenging technical constraints. Doing everything online and with low latency makes our work difficult, but in a good way(e)There are not many who do what we do.
Welcome to the end of the page, you made it!
If you like what you see, we'd love to hear from you, hit the button below! Or if you want to see more of the same why not check out the the full description? It even has pictures!
If you're not sure whether or not you like all of this: let us show you what it's like on the inside! We can set you up for a chat with anyone on the team to see whether this role is for you(e)🙂
If you're sure this is not the role for you, but you're interested in Ava: there are more positions open, take a look: