Very often we are required to provide system designs for projects in the early stages of the project life-cycle. As a concept, the path to these system designs is different than a system engineering path we would have taken as described in other posts, since the end result and means to get there are different. In this post I will try to cover what the path for prototyping system design should include and how it differs from a standard product development path.
A prototyping task may begin at many starting points: we have a lovely technical idea that we would like to present to an investor; the company we work at has set a technological roadmap and we need to start feasibility checks; we would like to test a second source for a certain element in an existing system; we need to setup a system for a next generation product in early or not so early stage of the product development; our team needs to check a patch for a special client; top brass executive is coming for a visit and we must show how beautiful our R&D department is (yes, I had that one as well) and so on. All these paths have two things in common: having to build a system with capabilities that did not exist before and probably having to get it done quicker than what it would have taken if we had taken the full path.
Quick, Not Dirty!
A word of caution. We tend to say that when dealing with prototypes we do things “quick and dirty”. In my humble opinion “quick and dirty” is one of the most dangerous pathways in the industry. We plan some kind of an experiment or a system, where we allow ourselves to neglect many aspects in the pretense of quick and dirty without noticing that by doing so, we eliminate our ability to get correct and clear results. For example, I once witnessed a feasibility experiment with an optical setup that reached a complete opposite conclusion because the setup did not include polarization elements which would simulate the final system. I realize that sometimes (many times) we have to get things done quickly, but let’s leave the “dirty” far away from our official research routines. If we want to get a feeling of something unofficially let’s have it done quick and dirty, otherwise, any experiment or a system that its measurements and results impact the system design of the product under development let’s have it as quick as possible, but please please, not dirty.
Back to prototyping. Prototyping is always fun! It is the part of the project where we create something new and accordingly our motivation and enthusiasm are at their peak. But we have to remember that prototyping should always come with a question or a set of questions to be answered by deploying the prototype. As system engineers or tech leads we have to remember that the questions to be answered are not always technical, but the prototype must answer the questions in a technical and, if possible, a quantitative manner.
For example, if we prototype a second source vendor for a servo motor in a robotic system the question to be answered should be something like “can the system with the proposed servo motor handle a load of 200kg?”, where the system will have the same bill of materials (BOM) except the servo motor in question.
Quantitative and Qualitative Proto Questions
I would like to dwell for a bit over the question, and how to phrase it to ourselves. Let’s divide the discussion into quantitative questions and qualitative ones.
Quantitative questions are the ones we get a measurable number from the system like time, speed, light intensity etc. In this case the formulation of the question to be answered by the prototype is allegedly simple. Let’s assume we would like to plan a fast high power laser metal welder, then our question would look like “what is the time that takes the laser to weld two pieces of metal?”. But wait, is that enough? Haven’t we forgotten something? Obviously welding 100mm of length would take less time than a 200mm length. But probably that’s trivial. How about the thickness of the metal plates? Does a 3mm thick metal plate behaves the same as 5mm? How about angles? Furthermore, do we plan to have this prototype in-house or sent to field testing? Unlike definitions for a product, where our aim is to have them as wide as possible to access a variety of markets, when prototyping we have to define for ourselves exactly what the purpose of the prototype is, otherwise the proto may become too close to the final product – which might prove to be critical if some of the project’s questions are answered too late in the development process. If we were to amend our question set it would look like:
- What is the time that it takes the laser to weld two 3mm thick metal plates in a straight line 200mm long in laboratory conditions?
- What is the time that it takes the laser to weld two 5mm thick metal plates in a straight line 100mm long in the customer’s manufacturing floor?
As we can see, we ended up with a set of questions instead of one. This is quite normal, since usually we have to cover a lot of subjects and scenarios when we verify if a technical feasibility exists for a certain system or technology.
The qualitative driven prototype questions are much more difficult to come by, mainly because the nature of the question we try to answer is more obscure. When we plan a proto to be presented to investors for example the nature of the question will usually be “can we make it?”. Can we make a robotic laser welder for the car industry? Can we make a tabletop microscope? Do we have a capability to deliver a technology that translates voice to text? The whole purpose of the prototype is to convince the potential investors to finance our product. So, at the end of the day the proto should indeed answer the question whether the team is capable of delivering the product.
The chart below shows some examples where qualitative and quantitative methods may apply.
Of course, when we formulate a question, we have to decide what is the pass/fail criteria or better yet, the go-ahead criteria. If we understand these criteria when formulating the questions or when designing the system, it saves us the “now what” awkward situation when we get our results and we do not know if they are good or bad. Obviously, setting pass/fail criteria is easier in quantitative questions.
For a prototype to be an effective one, it must answer as many questions as possible in our question list and hopefully all of them. There are however situations where regardless of what we think initially, the effort difference between prototyping and developing the product as a whole is little enough for us to skip the proto. We will retouch this point further on in this post.
Design The Prototype
As you have probably noticed, until now we were just figuring out what questions we have to answer. Once we have these in front of us, we may go on to design our proto. We have to remember that when it comes to prototypes, a system may be an optical breadboard with a simple light source and a lens, or it may well be a full-blown system with dozens of motors and multiple optical paths with tens of optical elements.
So, we start designing our proto. Our route will include more or less the same as a standard product development and design, with a few exceptions. The main difference I would point out is the stakeholders’ list – it is much shorter when prototyping. There are stakeholders that do not even enter the list like Supply Chain, Manufacturing, Customer Support and Regulation, Standards and ISO. These stakeholders’ roles are irrelevant for prototyping. Don’t get me wrong, if you have a Supply Chain ace in your team, you’re better off having a word or two with him/her but generally the disciples above are not required for this stage.
The following stakeholders are sometimes required in this stage: QA\V&V\QC\SQC\Testing (each technology field and company has a slightly different name for this team, but the bottom line is testing, validation and verification of the product. For simplicity I refer to this team as Testing) – if the proto goes to the field to a customer’s place we would want it to be tested and more so than what we would have done for a laboratory jig; Marketing \ Product – if any of the prototype questions are marketing or client related we would definitely need their inputs; Finance – well, that depends. Sometimes we have a Project Manager that will buffer the financial issues for us and sometimes we will have to coordinate all the financial issues directly with Finance. Regardless of how, we need to know we are inside our budget limits; Service\Tech Support – once again it depends on our prototype’s designation – if it goes to a client’s place, one way or another, we will need to install the system in an off-office site, and who knows better than Service how to get it done properly?
Eventually we have the technical R&D teams (SW, Electronics, Control, Integration, Mechanics, Algo, Physics & Optics, System) that we must consult with, but yet again, not with every prototype will we need all the technical team with us. An optical bench with a LabVIEW script will probably exclude the SW team, while a proto for a feasibility of a vision algo will probably exclude the Mechanics team. While sitting with each discipline, you will be able to identify the limitations and advantages you may exploit, very much like non-proto system design. One important thing to make sure of, exactly as in standard system design, is that you have all of the technical team together in the same room as our input from the brainstorming of all the team together will get us better ideas and approaches than we would have gotten from the divide and conquer approach.
Choose Your Stakeholders Carefully When Prototyping
When it comes to prototyping, choosing the wrong stakeholders may cause a failure to meet the proto’s goals.
A general note regarding consulting with the “elders and veterans”. By elders I mean the senior engineers, the senior team leaders, the consultants and simply the ones who have loads of experience and who have “seen it all”. First and foremost, if you have the chance, consult with the elders. Their advice may and probably would save you a lot of effort. But, a word of careful advice. Experience might sometimes be a curse. Not all advice from the elders is up to date. Technologies change and improve, access to knowledge is easier than ever, prices for gear and parts constantly change. What was an absolute truth 10 or even 5 years ago may now be completely the opposite. The seniors, myself included, may not be aware of these updates. Do your homework and once you are sure, correct us politely (or not so politely, some of us do deserve to be scolded sometimes). Do not go for a design or a technical plan that you do not fully approve of only because your seniors have advised you to. I am personally the main inventor of a patent over a technology that elders advised me was impossible.
And so, we are left with designing our prototype and building it with all the relevant technical teams. We have to do, as part of the design, a timeline assessment. As mentioned above, usually the prototype’s mere existence depends on whether it arrives in time for a certain event. We should verify that this timeline is met before starting purchasing the required gear. Such assessment may impact the design and throw us back a few steps in the process, even as far back as the system requirements themselves.
Some Do’s & Don’ts when prototyping
- Don’t skip the system requirements process – even if you end up with exactly one requirement, do not skip the process of actually sitting and writing the system requirements and going through peer review. You’d be surprised how many mistakes could be avoided when going through it. Even a small jig of an element’s spectrometric test should be with a line or two regarding what is tested and to what technical end with numbers.
- Do your technical budgets – do your technical budget also for proto systems. This particular “Do” is the only one in the list with an exception, we should only do it when we have a tight technical budget. If you are not sure how tight your budget is, it means you should have one done.
- Re-use as many existing subsystems as possible – you and your colleagues already know the ins and outs of the existing subsystems. If some of them can be used in your prototype, you get, for free, a working part for your system.
- Use off-the-shelf elements everywhere you can – that almost goes without saying. Custom elements, mainly in optics, have longer lead times, cost more and on top of that, do not always give us the required performance on the first batch. Use off-the-shelf if you can.
- Plan your system with as little number of subsystems as possible – well, that’s good system engineering practice in all system designs, but in prototyping it has an even more profound meaning. As an example, in a system where you need a laser with its control, if you could purchase the laser and its driver in one unit, do it. The integration difficulties of the laser unit with the controller unit are saved, which will get you faster to your goal.
- Consult with the technical experts as if you were planning the final product – yes, as a system engineer you must already have loads of experience and probably have your field of expertise, but a consultation with a technical expert may be the difference between a “just in time” prototype to a “late and not relevant” one. Let the experts help you design; they usually enjoy it as much as you do, this is a win-win situation.
- Refrain from purchasing elements with a lead time that exceeds 3 months – if there was a way to express pain via the internet, here’s where I would express it. When dealing with cutting edge technology and the latest and greatest of optics, mechanics, control and algo, we may be required to purchase one of those for our beautiful proto. These usually are products with less mileage in the field, with limited knowledge base, which may prove to be not exactly what we needed. Then to wait 6 months only to discover that the element does not fit our proto and then we have to repair or even order it from a different vendor is frustrating to say the least and may well be a killer of the project in terms of timeline and technology.
- Do not try to engineer the final product – we came to build a prototype, let’s build a prototype. If we start inserting more and more features that are needed for the final product, which do not advance us towards answering our proto questions, we will end up with a lot of technical issues that are off-track which would make unnecessary noise. That said, see below regarding skipping proto directly towards product development.
- Plan your resources ahead – resources, human or equipment, nearly always are limited. And yes, usually when product development runs in parallel to any prototype development the product development takes precedence. Secure any special resources ahead when you know your schedule. Nobody promises you will eventually get those resources on time (or if at all for that matter), but at least cover your end.
- Safety above all – I know we all tend to be more lenient in prototypes regarding safety but the most important thing in our job is to come home safely, without any visits to the emergency room. Sometimes we deal with lasers, moving metallic parts, high electrical currents, biological materials, nuclear materials and many other hazardous situations. Let’s make sure that no one gets hurt during the process.
When will we choose to skip the prototype
There are many times when the questions we ask for the proto are needed as part of a product development, and under the same project the proto has to be developed in parallel to the product. In prototypes with high technical complexity and which depend on elements which are already embedded in the final product we may reach a point that the proto will be a very close to the final product in terms of content and effort. In this case and at this point we have a very tough decision to make – do we continue to build our proto in parallel or do we answer our question list in the final product.
Like everything in system engineering, the answer is “it depends” as follows:
- Risk management – how big are the risks the proto came to answer, especially the technical risks. If the risks are big, you may consider pulling in the prototype at the expense of the final product delivery date.
- Effort & costs – when the proto development efforts are similar to the final product’s, we may consider skipping the proto.
- Resource availability – proto development is still a multidisciplinary system development. If we are short on laboratories, equipment and most of all human resources, we may decide to skip or reduce the prototype build.
- Timeline – well, obviously if the proto’s answer delivery date is close to the final product’s delivery date there is not much point having the proto. Thumb rule is ¼ development duration. For example, if the product development length is 12 months, anything closer than 3 months to the delivery date is already questionable.
In this post I’ve tried to demonstrate the initial phase of system prototyping, where the hidden traps are, where the initial difference between prototyping and final product development lie, and some good practices one may adopt to make the process easier and more effective.