An old irony in New York is the ubiquity of the ‘gourmet deli’. It is hard to find a deli that doesn’t proclaim to be gourmet. It is so commonplace that the word gourmet has lost all of its original meaning and perhaps taken on the opposite meaning. A similar phenomenon happened with the word engineer resulting in a dilution of its meaning. I recently heard someone describe themselves as a ‘domestic engineer’. While presumably tongue-in-cheek, the overuse and perhaps abuse of the term engineer has precipitated its loss of meaning and value. Other pithy examples include ‘sanitation engineer’, ‘sales engineer’, or ‘quality assurance engineer’.

Many will argue that title doesn’t really mean all that much and for the most part I agree. However being an engineer used to be more than a title: it also defined your role, your education, and your liability. I’m biased because I have a degree in Electrical Engineering and have gone through the rigors of the discipline culminating in the 8-hour long marathon Engineer In Training exam. One of my professors would state this disclaimer at the start of his Communications (where you design a wireless transmission network) class: “There are two things you won’t be able to do with a degree in Electrical Engineering: wire a house and fix a TV”. We all laughed at the time but it took time for the real message to sink in. His point was that engineering is a professional discipline that focuses on theory and design. We learned why things work and how to predict the behavior of systems we design based on first principles. It is not a technical trade that teaches you how to do perform specific functions. So we may not know how to wire a house, but it didn’t matter because under the hood we knew all the underlying theory.

The curious thing about engineering is that we often get confused as people who build things. The world of software has reinforced this stereotype, which I think needs to be addressed. At our core engineers are problem solvers: people who use mathematics, physics, and other scientific fields along with analytical skills to solve practical design problems. A civil engineer designs bridges and transforms building architectures into safe, viable structures. Electrical engineers design microchips, computers, wireless networks, power stations. Aeronautical engineers design aircraft. The list goes on and on. Note that in all these domains an engineer does not build these things as that is the domain of manufacturing or construction. Engineering is about design and prototyping that culminates in an explicit and detailed plan that describes how to build or manufacture the end product. The epitome of this philosophy comes from Qualcomm who revolutionized the integrated circuit (IC) business by focusing only on the IP surrounding the design of wireless ICs and eschewing manufacturing altogether. Other companies like ARM Holdings have since followed suit.

So where does that leave software? The challenge with software is that it is so widely adopted and the tools so easy that almost anyone can program these days (and in fact is now being taught to primary school students [1]). Knowing how to program is different from being a programmer, which is different from being an engineer. For example mathematics is taught throughout primary and secondary education but no one in their right mind would call themselves a mathematician unless they have a degree in mathematics. Most people I know with a math degree (including myself) do not refer to themselves as a mathematician. The same is true of physics: we learn a fair amount of physics but we don’t call ourselves physicists. The same really should apply to software: just because you know how to write software doesn’t mean you are a software engineer (a term I don’t particularly like). To be an engineer your understanding needs to be more than how to write software. You also need to understand data structures and algorithms, system design, resiliency and redundancy, analysis, numerical methods, etc. Without this foundation there is little guarantee that in production a system will operate at a level of reliability and accuracy that is expected of the system. This isn’t so different from a lay person building a house. While the house may stand and be livable, there is little analysis surrounding its structural stability, how much load it can bear, etc. I’m close to throwing out the baby with the bath water, so where does that leave us?

We are now truly in an information age, where even the most mundane products have become datacentric. This means that a product or service cannot exist without data to drive it. I’m not talking about Salesforce.com where you enter in data and retrieve it later. I’m talking search engines, mapping tools (Google Maps), recommendation systems (Amazon, Netflix), digital personal assistants (Siri), etc. I call this datacentric product development. Finance has been doing this for years under various guises. It is particularly prevalent in portfolio management, trading, and risk management, not to mention applications like creditworthiness or fraud detection. The term Financial Engineer embodies the role, which involves equal parts analysis, modeling, and development. These people are typically part of business groups and not part of the greater IT organization, which is usually designated a cost center. What is remarkable is that the role of Financial Engineer embodies this notion that being an engineer is about applying analytical methods to solve problems. The solutions just happen to be written in software. As datacentric product development becomes more widespread, being just a programmer will not be sufficient.

Organizationally I think this model is what technology product companies need to move towards. Product development needs more engineers and fewer programmers. What this means is that a single role must be responsible for understanding the business domain, assessing the problem, designing a solution, and illustrating that it works. The toolset of the engineer is more than just a Java IDE. It also includes working in languages like R or Matlab to analyze data and test hypotheses. In many cases this will also mean building a production system, while in other cases it may mean handing off to a vendor that specializes in building software. In some ways I’m trying to dial back the clock to the “good ‘ol days” but not for reasons of nostalgia. It’s really about recognizing what it means to be an engineer and what datacentric product development needs. (In a separate article I’ll share my thoughts on what I think an optimal organizational structure looks like when you adopt this model.)

The last question is what do we call this legion of skilled engineers? I’ve already professed my distaste for Software Engineer as I think this term confines the discipline to just software, which I’ve argued is only a small portion of the overall capability a person in this role needs. I could cheat since the organization I’m building is in the financial industry and stick with Financial Engineer but that still leaves an unnamed army to fend for themselves. Instead the emerging field of Computational Engineering [2] embodies the characteristics that I think datacentric product development needs. The reason is that the Computational Sciences are known for solving domain-specific problems using analytical and numerical methods, which of course includes writing software. People in these fields face the same challenges of those in product organizations: scaling, performance, reliability, accuracy of the model, etc. The term has not been widely adopted for Internet and business technology, but it is not difficult to see how it applies here as well.

If you’ve bought this argument and are interested in being a part of a new breed of engineer, don’t hesitate to contact me.

References

[1] Code Club: After-school group teaches children how to become programming whizz kids

[2] Graduate Education for Computational Science and Engineering