Today’s guest is Ilit Raz, founder and CEO of Joonko, which aims to build a more equitable workplace by automating the recruitment of diverse talent from underrepresented communities.
Joonko is an automated diversity recruiting layer named for Japanese mountain climber Junko Tabei, the first woman to reach the summit of Mt. Everest. You can learn about their talent pool, keep up with their blog, or check out their open positions.
ICYMI, read our blog post about how the recent tech layoffs have had a disproportionate impact on women, people of color, and immigrants.
Connect with Ilit on LinkedIn.
This week’s Lifeboat badge is awarded to pppery for their answer to Why use positional-only parameters in Python 3.8+?.
[intro music plays]
Ben Popper Hello, everybody. Welcome back to the Stack Overflow Podcast, a place to talk all things software and technology. I'm your host, Ben Popper, Director of Content here at Stack Overflow, and today I am joined by Ilit Raz, who is the founder and CEO over at Joonko. And we're going to be talking about a bunch of things that are in the news– specifically AI, natural language processing, and how tools like that can be used to enhance and increase diversity, equity, inclusion, and belonging. Ilit, welcome to the show.
Ilit Raz Thank you so much, Benjamin. How are you?
BP I'm good, thanks. So for folks who are listening, tell them a little bit about how you ended up in the world of software and technology.
IR So I'm originally from Israel. For those who don't know Israel very well, everyone in Israel serves in the army– men and women. I'd been serving in the intelligence unit in the army for eight years, doing AI stuff way before everyone thought it was cool. We're talking about 20 years ago when I joined the army.
BP Yeah, when we go back 20 years, that was before there were sort of these big breakthroughs where the ideas around neural networks and deep learning had sort of been validated. What kind of stuff were you learning 20 years ago and how did that take you towards where you are today?
IR Yeah. Compared to today, it was very basic stuff I would say, but it was still trying to look for patterns in massive amounts of data that probably most companies don't even have access to, even today. And at the very high level, the way that you think about it is, basically in the intelligence unit you want to take massive amounts of voice, text, you name it, and get the essence of it or a pattern of behavior that is going to give you an intel about something that's about to happen. So a lot of language processing, a lot of trying to understand the essence of a text– what is a text talking about? What are the main weapons they talk about? What are the main people they talk about? Those kinds of stuff. We didn't even call it AI back then. We were just doing it.
BP Right, called it just data analytics, analysis, intelligence, whatever you want to call it.
IR Exactly. And it was just data processing and stuff like that, but it was thousands or tens of thousands of data points on every second. So it's massive data that needs to translate into one identification of one event very quickly, very on the spot. And I think that's one of the main reasons that a lot of cybersecurity companies coming out of Israel are taking what they did for the intelligence unit and translating into the commercial world, and going ahead and doing this. So this is basically my first intersection with this world.
BP That makes sense. You're saying to take a ton of data and very quickly, as close to real time as possible, pull out the signal from the noise, identify what's important and in what way that's sort of actionable, and then go from there.
IR When you think about the intelligence unit, it’s very similar to what the US Army is doing or what they do in DC. They're trying to prevent something from happening, and sometimes this thing can happen within a week among unrealistic amounts of data and you need to understand the pattern of, “Okay, there is something going on here that hasn't been going on last week or whatever, and we need to know about it now.” I think there are a handful of companies that do really life-saving products, and for me, this was one of the really rewarding parts of being in there.
BP Right, right. So after your time in the military, did you go and work at other companies or were you always creating your own startups?
IR No. So I'd been in the Army for eight years, which is relatively rare. And I worked for a couple of startups. I moved a little bit from the engineering side to the product management side. My first job, not surprisingly, was in a company that works with the US government on the intelligence side, so it was very similar to what I've done before. And moving forward, I was doing more commercial stuff, the regular B2B, B2C, SaaS platforms kind of stuff but more as a product manager. And then seven years ago I started Joonko which brought me back to kind of working in the MLPA world.
BP So what was the impetus to start your own thing and what made you decide to focus on applying AI and LP to this area of diversity and inclusion in hiring.
IR So I never had a dream that one day I'm going to start my own company. It was just kind of the more boring stuff, like one thing led to another. But when we started the company, the idea was a very different industry. And when we started to meet people from that industry, it was around wellness, kind of like a wellness marketplace for the sake of the conversation. And we started to meet people from the wellness space and they said, “Why do you even do this? Why do you care about wellness? Why do you care about this?” And we were like, “Oh, it just sounds cool.” And people were like, “No, no. You need to understand. This is going to be a very long journey, a very hard journey. If you don't have real passion for what you do, you're not going to make it through the first year.” And we were like, “Okay, so maybe we shouldn't do a wellness thing. Let's regroup and think about something else.” And we decided that we are going to take a two-week break and come back with one topic each one of us is really passionate about and see if that clicks with other folks. And one of the topics that I brought in was, what if we do something for women and people of color? As a woman, both in the army, but also outside of it in the “regular world,” I was almost the only woman all the time, especially on the engineering side, on the product management side. When I did computer science both in high school and in university, there weren't many women around, and I started to learn the industry a little bit better in the US and I figured out, “Okay, this is big.” This is when Lean In came out first, the McKinsey report started to come out. I said, “Okay, this is a thing. People are talking about it.” And very straight out of the gate I was like, “I don't know what we are going to do but I know two things. I want to move the needle, something really meaningful. And the other thing, it has to be interesting for me from a technology perspective.” Because a lot of industries when they start, you take a manual process and turn it into automated, but there is not a lot of technology behind it. If you think about it like a job board or stuff like that, there is not really interesting, crazy technology behind it. And I wanted to be thinking about it as an engineer and as a product manager. I said, “I want it to be interesting for the engineers and product managers to come here not to come only because we do something that has a good impact, but because it's going to be interesting from a technology perspective.” So it was always important for me to find something that is interesting. I also think that only with real technology can you really solve big problems. Other solutions are a good step in the right direction but are not going to make a real impact. So this was, for me, super important. And after a long journey, we came up with what we do today, which is basically automated matching between candidates and opportunities, working in a way that you give and you get at the same time. So if I'm Stack Overflow, I'm part of the Joonko platform, every candidate that comes through and is processed that I decided not to accept at the end but got to the advanced stage, they're probably good for this position. It's not only good for me, they might be good for Stack Overflow too, because they got in the process enough. So at the end of the day, we take the best and finals from all the companies we work with and try to fit them to another opportunity, looking basically on three data points, which is the job description they originally applied to, the potential job description they can apply to, and their resume, and try to make the best overlap between those three to generate the best matches between companies and candidates.
BP Yeah, I mean, you make a great point. I've been on the hiring side a little bit as a manager and a ton of work goes into vetting people and in getting to late stage, you've done two or three or four interviews, you might be extremely qualified, but there’s somebody else who just happens to be a better fit for whatever reason. But that's a lot of really valuable data then. This person is somebody we would've hired 95% of the time, and so if another role like that opens up. You mentioned that you wanted two things: one, to be passionate about sort of the mission of the company, and two, to be genuinely interested in the technology. So talk to us a little bit about how the technology works. Is that a tech stack that you built in-house? Are you using models that are open source? How does it work?
IR Everything is in-house. There are two main areas that focus more on the AI and LP technology. One would be the piece where we develop an in-house decision algorithm for making a decision on each individual, identifying their gender, race, and veteran status, using about 8 to 10 different resources that we pull out data from, and then having a decision mechanism that we developed in-house. So this will be the more, call it ‘soft AI’, and then the other piece is obviously being able to make a match between two job descriptions and a resume. When you talk about the challenges around it, one is that a resume and a job description are written in a very different way. You need to make an overlap between them. And the second challenge is that two companies write job descriptions very differently, from writing about the company and stuff that we don't care about when we look at that stuff so we want to clean them out, and how do we even identify that this is a text about the company versus this is a requirement. And then understanding the requirements, which is a big thing. Because just looking at the title, a VP at Stack Overflow, which is a couple of hundred employee company versus a VP at Joonko, which is a 50 people company, is a very different VP. A director is a very different director and so on and so forth. And also the way that we look at a senior engineer is probably different from how you look at a senior engineer, and so the title is the very first step for us to understand the big cluster of in what space are we talking about? An engineer and a finance person shouldn't be in the same bucket no matter what the job description says. So we start with the very high level clustering. When I say high level, we have probably 10,000 clusters that we filter people through. And then we start going deeper into basically filtering out the job description. And the more challenging stuff on a job description can be startup experience, stuff that is a little bit more vague, that is not like five to six years experience in React. Stuff that is actually like good collaboration skills, good communication skills, and trying to figure out from what the people write on their resume, do they have those skills– yes or no, to some extent. So this is where it becomes really complex. We use multiple types of algorithms that we developed and techniques that we developed in-house. We basically train the model every couple of weeks, but also every time a new industry comes in. So we now started to work with a new company that we never worked with before and have job descriptions that we haven't come across before with requirements we haven't come across before. Our model needs to get used to those new things.
BP Right, right. Got to train them on the new ideas.
IR Yeah, on the new job descriptions, the new resumes that we haven't come across before. It can be a new industry, let's say medical devices. So they will have jobs that SaaS tech companies don't really have, and we need to help our model understand that a mechanical engineer and an engineer are completely two different things.
BP Right, right. So tell us about some of the companies that you worked for. What do you feel are the best demonstrations of this technology's ability to produce the desired outcome? And what does the ROI look like for a company that's investing in Joonko?
IR So, at the very beginning that's a note for the side. When we just started, one of our first projects was with Stack Overflow. We were trying to find biases in the data of Stack Overflow. Do people comment more for women versus men? Who is contributing to the community more, men versus females? So we had a big project that we did which was really cool.
BP Sorry, was this something you did with just our public dataset or something we worked on together as organizations?
IR No, no, we worked together. We got a bunch of data from you all, probably a couple of terabytes of data, and we just analyzed every individual user to try and really figure out how to make the community more diverse. I think this was the goal: can you give us signals on what we can do better to engage females within the community? And it was a really nice project. I think one of the things that came out of it was mostly around males chasing more rewards where women were chasing more communication, so how do you balance that and stuff like that. So it's been a very interesting project. It was right at the beginning when we were trying to figure out what do we really want to do. But basically we work with companies like PepsiCo, Walmart, Porsche, Nike, Adidas, and so on and so forth. We are in charge of about 15% of your underrepresented higher placements at the end of the year, quarter, you name it. And basically recruiting is numbers games. You said you had a little bit of experience as a recruiter or as hiring manager. It's a numbers game. The more diverse your top of the funnel is going to be, the more diverse people are going to come through. And the interesting piece about it is, the more diverse your pipeline is going to be, it will only grow, because you're not going to start looking at them as underrepresented folks, but actually as just folks. Or you're not going to be, “Oh, I have a woman in the pipeline. I have to uphold to it.” It’s like, “I have so many women in the pipeline that I can just pick the best one.” And we basically help people normalize the pipeline into a way that we look at underrepresented minorities as folks.
BP Right. That's an interesting idea because I do think that gets to one of the controversial things about hiring or college education, things that are up for debate at the level of the US Supreme Court: What does affirmative action look like? Should you be privileging one over the other? And you're saying, “Look, just normalize the pipeline so that you have a great pool of candidates from all the different groups and then pick based on who you think is best qualified for the role,” so that makes a lot of sense. So just as someone in the world of software and technology, but then especially as someone who works at an AI company, I'm sure you're keeping a close eye on all of the breaking news and sort of explosion of papers and new efforts to unlock capabilities through these large language models. Are those things that you can apply to your business or that you're hoping to sort of start to build into your technology suite?
IR Yes. We're currently more focused on things that are at the areas of non-biased decisions. This was a big thing over the last couple of years about how do we keep our data clean from biases to make the algorithm not continue making the biases that we have. Because at the end of the day, we do the matches between candidates and companies, but if we're going to rely on decisions human beings are making, we have to make sure that we somehow cut the biases out. So the fact that we only attract underrepresented minorities really helps, but at the same time, we constantly try to balance it with data of a mix of everything, just to make sure we are not too much leaning towards underrepresented minorities versus too much staying to the other side. So we really try to focus on that, because this is the number one thing that worries most of our customers– I want to make sure we are not in a risk that we're using a tool that actually duplicates our biases as human beings. And we really, really try to look at the numbers at the end of the day. What is the trend of the number at the pipeline? How can you improve it? But I think it's also helping us communicate with folks like yourself that are way more numbers-oriented. In general, I think it's hard to fight the numbers more than, “Oh, I think you should interview more women.” I think I interview enough women and that is becoming a ‘How do I feel about this?’ subject versus, “Hey, look, 80% of the women don't even cut the first step with you versus 75% of the men.” There is something here that we need to dig into and understand what it is, so we are way more focused on those kinds of stuff versus the new stuff that comes up. I'm sure we're going to come across that moving forward and they are going to be more interesting for our customers as well. So this is where our main focus is at.
BP Very cool. So helping to eliminate bias in the AI, and in doing so, ensure you have this really healthy and diverse pipeline and then people can make meritocratic choice and feel good about it. Awesome.
[music plays]
BP All right. At the end of every episode we like to shout out a member of the community who came on Stack Overflow and helped share a little bit of knowledge. Today, a Lifeboat Badge was awarded to pppery. “Why would you use positional-only parameters in Python 3.8?” Well, pppery has an answer for you and has helped almost 5,000 people, so we appreciate you spreading some knowledge. I am Ben Popper. I'm the Director of Content here at Stack Overflow. You can always find me on Twitter @BenPopper. Reach us with questions or suggestions about the show, podcast@stackoverflow.com. And if you like what you hear, leave us a rating and a review. It really helps.
IR I'm Ilit Raz. I'm the CEO and founder of Joonko. You can find us on every social media at JoonkoHQ. Thank you for hosting me.
BP Thanks for coming on.
[outro music plays]