Prototype Like You Mean It, Part 2

In part two of this prototyping series, Justin Davis and Greg Ross-Munro focus on what happens after you build the prototype: getting meaningful feedback, iterating with intention, and knowing when something is ready to move toward production. They explain how to run effective feedback sessions without getting distracted by surface-level opinions, why mental models matter more than button placement, and how fidelity should increase alongside certainty. The conversation also tackles a modern challenge: when AI-built prototypes blur the line between experiment and product, and how to decide when to bring in professional help.

Transcript

Copy Transcript Copied

SPEAKER_01 0:06

Welcome into Leap to Scale. This is the podcast that helps service companies take their business and turn them into more than a service company, more than just trading dollars for hours using technology to turn your services into products that can scale without adding headcount and also using AI and automation to efficiencize. Is that a word, Greg? Make more efficient. Make your internal processes more efficient so you can lower your costs and increase your margins. My name is Justin Davis. I'm the vice president of user experience over at Source Toad, and I'm here joined by my friend Greg. Greg, good to see you.

SPEAKER_02 0:45

Nice to see you too, Justin. Last time we spoke, we talked about prototyping and turning prototypes into, you know, you know, using how to be quick about building prototypes, how to pick the level of fidelity that you need in your prototype, uh, using spreadsheets, I believe we talked about um as like a phase one for your prototype, and maybe you've already built something in a spreadsheet, you can monetize that. And then we also talked about how AI is kind of changing the prototyping kind of conversation in general. I think we we mentioned a few things last time we we we chatted about um topics that we need to like kind of maybe talk more about to wrap to like round this out. Um and I think the first one maybe that we should talk about is about how to get feedback on your prototype and like why that's important. You I know, as like a UX guy, have done a lot of user feedback tests, right? Like feedback sessions. How many do you think you've done?

SPEAKER_01 1:54

Or like user interviews. It's definitely in the high hundreds if it's not like over a thousand. It's probably not over a thousand realistically, um, but it's definitely in the high hundreds. Um because each one of those were like 60-minute conversations with people. But um yeah, hours and hours and hours, hundreds of hours.

SPEAKER_02 2:13

Um you've done way more than I have. Way more than I have. I mean, I've I've done I've done user feedback for prototypes and things in um a lot of kind of what we like what you call hallway testing, where you just grab somebody out of the hallway and you're like, hey, can you like look at this thing? And like, what do you think? Um and I've done a couple of sessions, especially like in the travel tech world where I was building something for a particular client. And um I was in an airport in France once, and I designed this uh uh TV system for a client and a remote control, and I had I had the uh the remote control in my bag, and at the airport I saw a lady with a bag tag who was clearly going to on an experience on a journey with um my customer with their with their travel company, and I was like, Oh. So I pulled up a chair next to her and I was like, hey, I know this is gonna sound really weird. Um I'm building software for a blah blah blah travel company. I build a TV system for them. Uh would you mind would you mind like testing it for me? And she was like, What? And she was a she was a lady in her uh 70s, and we talked a lot about I learned a lot about how furs have to be stored correctly. Like this was but I also gave her the remote control and I didn't say a word and I watched her use it, and I learned more in those like 15 minutes of talking to this lady at at Charles de Gaulle Airport than I could have ever learned, like iterating a hundred times in my office.

SPEAKER_00 3:54

Yeah.

SPEAKER_02 3:55

How do you do it? And and who should you do it with?

SPEAKER_01 3:58

Yeah, I mean, I yeah, I think we you know it's funny, I I tell people a lot that like because sometimes people get intimidated by this and like, oh man, I gotta get like this user feedback. People people think like, oh, I gotta go like an eye tracking lab, and they sit behind a double mirror and watch them use this thing, and it's crazy, right? Tell people to do exactly what you did, which is just like go to somebody in the coffee shop and be like, I will buy you a coffee if you'll sit with me for 20 minutes and chat with me about what I'm building because I'm just I want to get your feedback, right? And so I think that's a fantastic example of how easy this can be. And the and kind of to set the stage for this, in order to kind of answer that question why why do you do this in the first place? Why why do you care, right? I mean, as dumb as it sounds, right? Like in the previous podcast, previous episode of this uh prototyping podcast, we talked about how to create something. We talked about how to get an idea out of your head and get it into the world. And fundamentally, what that is about is having an assumption uh about some some problem that you think can be solved in some type of way. Um, and uh in particular, especially in the business uh context. And so now what you want to do is you want to find out if you're right, because most of the time actually we're wrong, but we're hopefully wrong in some way that is we're wrong in the right direction, right? And I think that's what you're trying to find out is am I wrong in the right direction, or like, am I wrong in the wrong direction? Because you're probably wrong at some to some degree, right? So you want to get this in front of people as quickly as possible so that you can find that out. And this is why I'm just I'm a huge fan in general of telling everybody about what you're doing and building and showing it to you, because no one's gonna steal your stuff, right? Go tell the world about what you're doing. So, anyway, when you are doing this, you built this prototype, who are you looking for? I mean, the first thing is to do exactly what you said is who am I building this for? It has to match your target audience. We're not trying to go get anybody, right? What we're trying to do is find somebody who looks like the person, not physically, but metaphorically speaking, looks like the person who is going to be using this product at um at the end of the day. Um, and so your your experience of being in the airport and being like, oh, this person like matches the profile of my target user is exactly that, right? Um, so that's the first thing you want to do is think who who can I get to look at this, right? Um, and and that might be internal folks. If you are doing, for if you're if you're doing process improvement stuff, right? You might say, I need to go sit down with the director of HR and make sure that like this is how they want to give me the data. Because often what you'll find as you go sit down with them is you'll find out, oh, you know, they actually think about this problem a little differently than I think about it. And they want to give me information in a different way than I thought that I would receive, right? And those handoff points are really, really important. And so sitting out, and then you will have those same conversations with like external customers. If you are working on productizing your service, one of the best things you can do is go back to your existing customers you've had before and ask them to try and look at this new productize version of what you're building, and they'll give you a lot of really great feedback too.

SPEAKER_02 7:14

Okay. So I have a question then. Um the Henry Ford famously once said if he had asked people what they wanted, they would have told him they wanted a foster horse. I don't know if he famously said that or if that's like something that's just attributed to him, because I'm not gonna look it up. But uh it sounds good. Um but I know what I mean. I know what he meant, or what uh the quote means, which is that like a lot of the time the when you do use a feedback testing, or when you like show somebody a prototype and uh you uh have an idea in your head, a vision, and because you're kind of building something or you're the expert, it is it's tricky to get that vision across to someone, and somebody else cannot see what's in your head, and maybe they won't see what like the vision is down um down the line, especially if you're like doing this iteratively. How do you make sure that they don't tell you they want a faster horse? How do you get them? Maybe this is a difficult question, but like how do you get them to like tell you the feedback that you need?

SPEAKER_01 8:19

They are yeah, yeah, yeah. So I think there are two guidelines that I would give you on on sort of staying on the rails there. The first thing is to stay focused on the hole, not the not the drill, right? You're focused on what the result is, right? You're not focused on what the tool or the means to the result is. And this is a this takes a lot of discipline because for most people, we are problem solvers and we latch ourselves onto concepts and solutions and ideas first, right? And it's harder to think about problems and to think about problems, especially to think about problems and think of other solutions to which you've already solved it. That's like a painful, hard thing to do. So we tend to latch on to solutions, right? And so, but the more you can train yourself into thinking about problems and what the what the what the pain point is is that is being solved, not necessarily the way it's being solved, and and ask questions around that. Because what you're actually trying to do is not have somebody tell you, oh yes, this solves my problem, or no, this doesn't solve my problem. What you're trying to do is have them essentially, well, you are at some point, but you're also trying to essentially get them to give you enough context and information to where you can make that leap yourself. And you can go, oh yeah, like you've told me, like it's obvious that like we are we are doing the same thing here, right? And they don't have to direct your project and tell you what to build. You get to do that, but they you can verify that their idea of the world is the same as yours.

SPEAKER_02 9:49

Now let's talk about Wait, so saying that so you're saying uh user feedback should not be uh they're like, oh, it would be better if you put that button over here. That's not what you're looking for, right? You're looking so that you're asking them these questions so you can see how they think and how they work and their view, their view of their job or their whatever that you're trying to like duplicate, solve, automate, etc., so that you can replicate that correctly in your own head and as a result reflect it back into the product.

SPEAKER_01 10:21

Right. Where most products I think die, and where most of this stuff dies, is not because the button was in the wrong place or it was ugly or it was whatever, right? Those are not actually the reasons, those are the easy things to point to. The reason why most of these things die is because the mental model of the problem of the person creating the thing does not match the mental model of the person for whom they are solving the problem. And it is at the very core foundation, they have a different way of looking at the world, and that is expressed through a product. I just read a blog post earlier today called Um, like Your Data Model is Your Destiny, that talks about how the way you think about concepts and ideas and how they fit together in a product or a service or a thing, um, is like defines kind of what your product is and is like your unique advantage and that kind of thing in a different context than what we're talking about here. Um, but I I think it underscores the same idea, which is that you have to make sure along the way, yeah, you're not talking about colors and fonts and button shapes and all of that kind of thing. What you're talking about is is my understanding of your problem the same as I think it is, and help me to make sure that we're on the same page conceptually. Because in in if you think about it theoretically, and I don't know, this is kind of a deterministic, this is actually a deterministic way of looking at it, which is probably not the right way to look at it, but um with the advice anyway, go for it. If everybody had the same exact perfect context, you would converge upon pretty much the same answer, and you wouldn't have to check the answer because the context would lead you to the same answer. Now that's in a deterministic world, but let's just assume that it would lead you to a relatively narrow set of answers, right? So then if that is true, then the whole goal of doing this and talking to the people, especially early on in the process, before you go too far, is to say, do we share the same understanding of what we're trying to do with this thing? And let's make sure all the way along the way that we maintain that shared understanding of that thing. And I think that's the most critical part here. And so, like, you want to, and that gets to the other part that I was gonna mention, which is the second kind of guideline here, which is you want to make sure that you do rounds of feedback as you're building anything and prototyping anything, you want to make sure that you do rounds of feedback in successive loops of fidelity. And what I mean by that is that um I say all the time that you're the fidelity that you're working in should represent the level of certainty in your thinking. Meaning, if you have a general idea, you should be sketching it uh and and um and just kind of like back in the napkining because you're still trying to form up the general shape of it. You shouldn't be worrying about details like thoughts and buttons and colors, right? Until you vet that like the shape of it's correct. And then you add on more layers and add more layers, just like an artist adds on layers and layers and layers onto a canvas, right? And so as you do that, you want to get feedback on every layer. So the first layer is like the conceptual does her idea match, and eventually you'll get to the oh, it should be green or it should be blue conversation, but that's like round 10 of feedback, not round one. So make sure you're having the right conversation at the right time.

SPEAKER_02 13:42

And you also talked about like the increasing levels of fidelity with that, right? Yeah. So when you when you say like having the right conversation at the right time, when you build a prototype, do you build like five versions of the prototype? Do you like what when you say fidelity, what do you mean there?

SPEAKER_01 14:01

Yeah, yeah. Uh that's a fantastic, um, it's a fantastic question. I mean, I think when I think about fidelity, I think about the level of um finalness in the thing, basically. Um, and broadly because I think in terms of product design uh and that sort of approach to things, I think in terms of like a conversation is the first fidelity, right? Like, hey, I've got an idea. I think this. What do you think, right? That is that is a mental sketch, right? If fundamentally at that level, the person goes, no, that won't work at all. And here's the reason why. Well, it's a good thing that you didn't just go spend hours and hours and hours building something or sketching something or whatever it is, because you started from the wrong point, right? So you have a conversation, you say, I'm thinking that this would be a cool thing to do, or I'm thinking this might help. Can you give me some feedback? Tell me why it won't work or what am I missing? Ask the two questions that I love asking, which is what is wrong and what is missing from what I'm telling you right now, right? And that will help people to find things. Because often people will be like, Oh, yeah, it sounds fine, right? But when you give people handles to grab onto, they'll be like, Well, here's something you're missing, or here's something that I think might be wrong. And so um, that kind of opens up the the space for feedback. Um, as you so it starts with a conversation, then it goes to a sketch or it goes to a wireframe, right? You might make something in Figma if you're creating a product or a service, or you might prototype something on make.com. And then you go, okay, well, this is kind of working now, but man, if we turn this on for the whole company, it's going to probably break at some point. So now we need to go like make this into a real web app, right? So now we're like stepping up in Fidelity. We've already proved it out, right? But now we make a jump. And and in some instances, you have to burn it down and start from scratch. And actually, in a lot of these, this is called creative destruction, you that happens. You have a conversation and then you kind of burn it down-ish, and then you start over with a sketch, and then you burn that down, you start over with a wireframe. You don't turn a sketch into a wireframe, you start a wireframe from sketch, right? So every iteration actually does burn it down kind of from scratch, and you get a chance to create it anew. When you go from, say, like a prototyping tool, like an AI prototyping tool into production, we start to have an interesting fork in the road where we say, well, you know, maybe we could take this thing you already built and prototyped, and we could sort of like ease it its way into production if it was built well in the first place. Or maybe we just have to kind of again do the same loop. We just start over again, build it anew, but following on all the successive destructive iterations that have led us to the same vision, right? And that's what's that's the value that's being created out of each iteration, isn't the thing, it is the certainty. Um, that is the that's the the value of each loop, basically. Okay.

SPEAKER_02 16:49

But but then I I have a I have a question for you. If if you are if your prototype is basically like almost a part of your final product, right? It's part of like the product development lifecycle. Yeah. Where you know you're constantly like burning these things down, you're constantly building them, you're showing them to people, you're getting feedback, you're interpreting the feedback, you're learning the world more and more, your fidelity gets higher and higher. Uh it's and you're you've already kind of like jumped a little bit into saying that now maybe I want to take this thing and I want to that's not that's no longer a prototype, and I want to actually like start polishing it and like making sure that it works in the real world. But I guess my question is where's the line between prototype and product, right? When it's when we're talking about software. I think it's an easier line to to to kind of try and cross if it was like a physical thing. Right. But that can get real blurry when it's a digital product, right? Like how do you do you have like any rules about that? Or those rules changing?

SPEAKER_01 17:55

Yeah, those rules I think are changing. I mean, I actually am starting to think that there's in some and I think that there is not a hard and fast rule of physics that says that a prototype is different than the product itself. And I don't think that they have to be, I think what they can be thought of is a life cycle, a part of the life cycle. And it might be maybe maybe a better example is like a an in like some kind of insight reference where it's like some kind of like metamorphizing that you go through, and like in some this is a lava phase of my software. Right. I mean, and then like some like molt off, and then some like go into a cocoon and completely like rebuild like from scratch, right? And so I I think there's actually a new mental model of like what a prototype is. I think when we think about a prototype, typically we think of something we will throw away and build something new. But I think that the AI tooling, things like replet levelable and some of the vibe coding platforms are changing that and will continue to change that more and more. And so then the question I think becomes well, okay, if that is true, and if now the line between prototype and product gets blurred, then what would you need to do to your thing that you make in order to get it into the world so that other people can use it? So that and then and I think that often when we think of prototype versus product, we think of like a prototype is a thing that one or two people use, and then a product is like a thing that like greater than handful-ish use, right? Or some like general uh heuristic like that. So um I think yeah, I I remain optimistic actually that it's gonna get easier and easier to go from prototype to production. I do still think that there is a gap that you do get to where a lot of these tools, but these tools, especially the vibe coding tools, you know, the replets and the levelables of the world and cloud codes and all that kind of thing, have made it really easy to build stuff and explore ideas. And to get like really far, actually. There is still like a little leap right now, and I think right now it has pushed the line between prototype and production further out, but there does become a leap where it and I it's more of an infrastructure and configuration leap now.

SPEAKER_02 20:28

Um almost more than security and right.

SPEAKER_01 20:32

So it's like, is it secure? Is the code written in a way that's maintainable and and can scale? Um, both of those things, which I have other opinions about as well. Um, but they are important things to think about in a lot of different situations. Um, and also like where are your databases at? Like, what is all the infrastructure like? How do you get all that set up? That's not something you can easily just type into a VIP thing because it's writing code, it's not building infrastructure. And I think that is uh an important right now, that's where the line seems to be moving to.

SPEAKER_02 21:02

All right. So then, I mean, I'm not trying to say that you and I are gonna have jobs forever. Um, I would I would like to think we do would, but um hopefully hopefully we offer something else other than just being able to code. But uh if you know, like I don't know, I I think I think about we went to go uh meet with a CPA firm the other day. They want to build um a system for their clients where they can like log in and they get a whole bunch of reporting um done to a specific type of CPA firm. And um it's very, very cool. What they they've like sketched some stuff on the whiteboard and everything. And I was in a position where I was thinking like, huh, you know, we could teach them how to kind of prototype this on their own and then bring it to us when they're ready to um to like you know turn it into a real product, you know, to like add. There are CPAs. It needs to be like bulletproof, right? Like if you've got people's financial data, it's connected to banks. You don't want to vibe code that.

SPEAKER_01 22:07

Please do not vibe code that. Than like a little cute social media network.

SPEAKER_02 22:11

But like they we could they could build like the front end and like see they could show they could show it to clients, maybe, and then they could they could click it. It could be fake data. There's a quite a lot they could do on their own. They're busy, but um, so maybe theirs my value is our value. It's like taking away some of that you know busy work for them and like getting them stuff that is high fidelity quicker. But like, do you where do you where do you think you should you know think about doing it yourself versus like going out and hiring a pro? I mean, this is kind of like the build versus buy, but also kind of versus like assemble kind of argument or question here?

SPEAKER_01 22:48

Yeah, I mean I think that I think there's a few things. I mean, I think that one, I think if you're building relatively small, non-public facing tools internally that are not dealing, like not destructive in nature or like potentially compromising in nature. Um, I think that you're safe mostly to experiment um internally, build things, see what you can come to. At some point, you might realize, oh, this could be used by other people, and I want to productize this and spin this out as its own product. Then you probably want to bring somebody in who knows how to build an app really properly to grow and scale to that type of situation. Um, I think that also if you are prototyping something that you are considering using long-term that might touch customer data, might be few front-facing, might be sensitive, or might just be mission critical. That, like, oh, if this thing in the future we build a thing, we prototype a thing that goes down one day, then everyone might have to revert to this old way we were doing it. And man, that would be a horrible situation. Then it might be advisable to get somebody um alongside you who sort of knows how what the entire prototype to production pathway looks like, just to make sure that you have the guidance at the start to know how to set it up right so that you have a much easier path. Because what you don't want to do is like build a thing and then be like, holy shit, this is working. I need to get this like properly stood up because these things are melting, or like it, I'm getting charged all these usage fees on on Replit or whatever it is, and I need to like move this thing to a proper server, and then you go, oh uh oh, but it's like now six months of migration work because we built this big thing in a way that is not necessarily compatible with getting out of the real world well. Um, so that that's a a thing you can do uh as well to help with that, you know.

SPEAKER_02 24:49

Yeah, so I mean, I think that we kind of covered almost everything we wanted to talk about, right? You know, we I think the big my big takeaways, or I think that what people should leave with is if you're gonna prototype something, don't over uh don't overthink the user feedback. Don't don't don't try and like don't don't overdo it. Most of the time you're trying to find out if your prototype is going in the right direction. And like you need to adjust based on who you're talking to. And like what you're trying to get is what's in their head into your head, rather than like where the buttons should go. And then, you know, if you we you we used to talk about prototypes as these things that we kind of would like build to test with and then throw away, or maybe that like one or two people were using them, and then we would switch to building things as uh you know, now it's part of more like one step along the product lifecycle journey. And uh it's if you're gonna be building something, anything that is like complexity or like security requirements or sensitivity, maybe better if you have like a partner alongside with you. But if not, you can probably do a lot of this on your own. So I'll I'll leave you with the last thing. If you had to give a quick little playbook on what to do, on how to build a prototype from beginning to end, what would you know, in in a few words, what how would you what would you say?

SPEAKER_01 26:15

Yep, I think the I think it starts with this pick one thing that you want to prove, one problem you want to prove, one assumption, one thing that you want to prove right. I I bet you we can increase this by doing that. I bet we could decrease this by doing that. I bet we could make money by XYZ. Pick one thing. One thing. Choose the lightest way to prototype this. First it's a conversation, then it's a sketch, then it's a spreadsheet, then it's an app. Don't jump to fulfillity too fast. Test it with some people early and often. Go through multiple rounds, tell people about it, explain it to people, sketch it for them, and do this in iterative rounds. Make sure to share it with like maybe like two, three some odd people at the beginning. As you get later down the line, you can test it with more people if you need to, especially if you're going to production to large audiences. But if you're internal, a couple people, two, three people, just to get some gut checks. Always a helpful thing. Um, measure real outcomes, make sure it's actually doing the thing. Make sure that like you really know what you're trying to improve and track to make sure that you know that you are actually getting the result that you need to do. And then figure out if you're going to keep that thing, if it's working for you, if it seems to be proving out to be a correct thing, or if you're going to kill it, reinvent it, make something else, or if you're going to take it into the world and make it a big deal.

SPEAKER_02 27:35

I'll tell you what, it it hurts to kill projects, but one of the best decision decisions you can make is by saying, Hey, we have the money we have invested up until this point to learn that this is not something that should exist is way better money spent than building something that costs 10 times as much that no one uses. Yes. So as hard as it can be, sometimes you have to kill things.

SPEAKER_01 28:02

Yes. You just have to go around drilling holes sometimes to see if there's oil. It just don't drill the whole well, just like drill small holes real quick, and then once you find some oil, then you can drill a big well. We've been doing this since like forever.

SPEAKER_02 28:15

Well, thank you, Justin. It's uh it's Friday again.

SPEAKER_00 28:19

Yes, it absolutely is. And I know that you're about to go get uh beat up, so enjoy getting uh beat up this afternoon.

SPEAKER_02 28:27

Yeah, uh judo, fun, good times. I'm hopefully not good to be enough. Well, thank you very much for talking to me about prototyping again, and thank you guys for listening. And we'll see you next time.

SPEAKER_00 28:37

All right, we'll see you guys. Bye.