Interviews are a two-way process, and it’s important that engineers assess the prospects of joining a team in terms of their career growth, mentorship, or whatever they want to optimize for.

Inspired by this blog from Julia Evans, as part of my interview preparation I built a list of questions that I occasionally ask AI companies when interviewing for machine learning roles in order to get a better sense of what their product is about,  what might be my responsibilities, work environment, culture, engineering craftmanship level, and the technical expertise of the team that I might join.

This will be an ongoing list that I will keep adding to, and improving as I grow in my career.

You can contribute to the list on this GitHub page here, and excited to hear any feedback from your experience in the field!

Machine Learning Interviews Questions

1) How long does it typically take to get from experimentation (model staging, training, testing) to production?

2) What level of automation does your company have for training and deploying models?

3) How do you manage problems like data shifts or model shifts? (No clear answer here like unit tests is a bad signal)

4) Do you own the whole MLOps stack? Are any parts of the pipeline outsourced?

5) Can you tell me more about your data versioning practices?

6) How often are your datasets updated? (Weekly, Monthly, Annually, Metric-based)

7) After serving your end-customers with ML products, how do you track model performance?

8) Are you using containers for ML? What for?

9) Are you using Kubernetes for ML tasks?

10) Where do you mostly run your ML workload? (Cloud, Hybrid, etc)

11) What does your company use for MLOps? (MLOps platforms vs. In-house developed tools)

12) What cloud providers are you using? (If any)

13) On average, how much are you spending per month on cloud costs (USD)? (<2k, 2-10k, 10-50k, 50-150k, 150k>)

14) Where is your data stored? (Cloud, Local Disk, Other)

15) What is the typical size of the datasets you use for model training? (10MB, 10-500MB, 500MB-1TB, >1TB)

16) What is the most common type of data your company analyzes or uses? (Tabular, Text, Images, etc)

17) Tell me about any technical debt the team has taken while building your ML system? How did you pay it off?

18) What use cases has your organization benefitted from AI? 

19) What types of GPUs are you using? Any techniques to maximize GPUs utilization?

20) Any growth engineers on the team? (If it’s a MLOps platform company having such engineers would be useful)

21) What is the average CPU/GPU/Memory count per task that you or your team use for running ML tasks?

22) Any internal tools built by the company to make developers lives easier? (Especially in the ML context)

23) Are you providing memberships to engineers to use GitHub Copilot? (That would be nice)

24) What points are the most challenging for your team? (Data sourcing, Cleaning, Training, Deploying, Monitoring?)

25) Does your team have model re-training and monitoring pipelines?

26) What is the average memory count per task that you or your team use for running ML tasks? 

27) On average, what is the size of the models that are being deployed? 

28) What tools are you using for preprocessing/ETL? (Spark, Hadoop, etc)

29) Tell me more about the models you use? Open source or proprietary?

30) Does your team embrace or utilize online learning to improve model performance?

31) Can you talk about what techniques you use for labeling your data? Any platforms?

32) Where do you host deployed models?

33) How fast is the average inference speed of your models? Any latency restrictions that you try to optimize for?

34) How many models do you currently have in production?

35) Do you leverage data streaming for your models?

36) Any research scientist on your team? Can you tell me more about their background and what they work on? (Nice to assess who’s working on training these models)

37) Are there any platform or solutions engineers on the team? (Ofc if the business model requires so, ideally always looking for reasonable ratios of people with respect to responsibilities).

38) Can you tell me more about growth opportunities in ML at your team? For example, what are you doing each year during NeurIPS? Do you have reading sessions for papers? Maybe tech conferences?

39) Any public speaking opportunities or ways to promote engineers work within the organization?