This is what it really takes

1. Solid software engineering is paramount

Building a truly self-driving car is the moon landing of our time. The task is nothing less than building a driving robot that delivers super-human driving safety in highly complex environments full of vulnerable, hardly predictable other agents while facing an infinite amount of edge cases.

This degree of complexity requires quality software engineering and great organization, two things that may fall short when trying to push forward in the highly competitive field of self-driving. Research scripts and glue code won’t suffice for a reliable product, though. The number of potential error sources in autonomous vehicles (sensors, networking, concurrency, …) is striking, and even crippling at times. To keep integration issues, software crashes and eventually disengagements at a minimum, software for self-driving cars must be designed for reliability.

A self-driving car engineer is a specialized software engineer, and one of the core responsibilities for this role is to ensure a high level of software quality.

High quality software means:

  • Solid software design
    Well-defined and consistent software architecture is in place; design patterns and coding best practices are followed where applicable; code is modular, reusable and concise.
  • Embedded software development
    Memory management, timing, concurrency are handled with care; computational complexity and runtime bounds are considered for algorithms.
  • Test coverage
    All product code reaches a reasonable test coverage before going to master. Unit tests, smoke tests, and integration tests run in CI for every build and keep the code base robust.

So what can you do in case you are not at this level yet?

    • Contribute to open source projects
    • Get work experience in a domain with less severe constraints
  • Get a bachelor’s or master’s degree in computer science

2. Self-driving car engineers are specialists, not generalists

It is great to have an overview of different technologies for self-driving cars such as object detection, SLAM, end-to-end driving, etc. – however, while generalists make sense in startups, bigger companies working on self-driving car tech need specialists who excel in at least one domain rather than generalists who can do everything somehow but nothing great. It should also be noted that building self-driving car software does not only involve functional engineering but to a much larger degree the implementation “enabler” technologies like parallel computing.

Here are a couple of competences that are highly important for engineers working in the self-driving industry (non-exhaustive, not ordered by priority):

    • Localization, SLAM
    • Computer Vision
    • Software architecture
    • Statistics, probabilistic and time-series prediction
    • Machine Learning and Deep Learning
    • Databases: relational and NoSQL
    • Sensor technology: camera, lidar, radar, ultrasonic
    • Vehicle kinematics
    • SD and HD Maps
    • Simulation and real-time computer graphics
    • Real-time processing, parallel computing, optimization
    • Functional safety
    • Build systems
    • Software testing and test-driven development
  • Automotive background

If you are great at two of those domains AND you have a solid software engineering background (see above) you should be set for a job as a self-driving car engineer!

If not:

    • Get work experience in a domain with less severe constraints
  • Get a master’s degree or Ph.D. in any related domain

3. Applying Deep Learning
≠ understanding Machine Learning
≠ being a great self-driving car engineer

Deep Learning is overly hyped and makes up for only a fraction of the work necessary to build a self-driving car. In fact, it is easier to find engineers with a solid Deep Learning background than with a solid embedded software engineering background.

It is important to introduce this down-to-earth understanding of Deep Learning before emphasizing that it is indeed a key technology without which cars can’t be realized.

But still: with online tutorials that spoon-feed applied deep learning, pretrained DNNs available online, frameworks that turn training, inference and even deployment on embedded platforms into few lines code, the discrepancy between those who understand the technology in-depth and those who can merely apply it has never been more striking.

Going beyond related work

You can get quite far with this “shallow” knowledge, but only to the point where related work (papers, code) is available. Despite the thousands of existing papers on Deep Learning available online, there are still areas of autonomous driving that are barely covered by public research.
If there is no code available for a paper, you should be able to implement the proposed idea from scratch. And if there is no paper available it’ll be up to you to figure out how to solve the problem at hand.

Not being familiar with the theory behind Machine Learning makes it very hard to debug training code. If a network doesn’t train and the only options you have are changing the data and changing the architecture, you’ll often be stuck.

What can you do to get that in-depth understanding of Machine Learning?

    • Mindset: Actively invest time in Machine Learning and Deep Learning theory, look up all these ML terms that you have heard of, but never took the time to really understand:
      Batch Normalization, Vanishing Gradient, Backpropagation, Gradient Descent, LSTM, …
    • The above mentioned online courses, but choose the ones that are about ML theory, rather than about ML application
  • Get a degree in Machine Learning

Being “good” in Python

Python has established itself as the go-to language for Machine Learning. Python features a simple, concise syntax and offers an abundance of libraries, particularly for scientific computing and Machine Learning.

It is a pity though, that Python is often used bluntly as a means to an end to quickly apply an idea. Mistakenly assuming Python is trivial and disinterest in Python as a language lead to poor understanding of its proper usage and consequently to large script-like code piles that become impossible to maintain, are eventually thrown away and finally replaced by a new pile of python ML code.

Python is among the domains with the biggest gap between self-assessed competence and actual competence, which can lead to awkward moments in job interviews.

If you are not sufficiently familiar with Python:

    • Mindset: Actively invest time into Python as a programming language, strive to master the language and write beautiful, pythonic code. Leverage object orientation and build reusable modules.

4. Automotive software development

While a startup can shield itself from automotive software engineering constraints for quite some time, there is no way around AUTOSAR, Automotive Spice, Functional Safety / ISO 26262, vehicle bus systems, vehicle development cycles etc. when targeting product-grade autonomous driving software. Not all of these constraints will affect you as an individual engineer, but it is hardly possible to avoid them entirely. A background in automotive SW development will make your life as a self-driving car engineer easier. If you don’t have an automotive background, you’ll have to get familiar with some of the foundations on the job.

In any case, “Automotive software development” is not only extra theory and processes to be familiar with, there are a couple of very practical implications it has on your work:

Software development innovations take time

With its long product cycles, the automotive industry is not necessarily up to date with modern software engineering practices. DevOps, Scrum, Continuous Integration, Test Driven Development, etc. are certainly not as widespread as they are in IT. The reason for this is not necessarily car companies being slow at adapting to innovations, but rather the fact that it can be quite hard to map these practices to the existing automotive engineering processes with their internal and external dependencies, long lead times, backward compatibility, decade-old legacy code, etc.

Automotive software is fragmented due to outsourcing

For established car makers, outsourcing of engineering tasks is widely established, particularly for software. It is common that OEMs rely on a pyramid of suppliers. The OEM integrates a Tier 1 supplier’s ECUs (Electronic Control Units, i.e. car computers) including the base software on it. The Tier 1 supplier integrates software and hardware modules from (usually smaller) Tier 2 suppliers. The Tier 1 supplier often shields the OEM from the Tier 2 supplier until the point where there is no direct interaction between OEM and Tier 2.

In practice, a Tier 1 could supply a radar system including the radar sensor, an ECU for data processing, and base software. A Tier 2 supplier could supply a software module that detects and classifies objects in the raw sensor data. Another Tier 2 supplier could supply the transmitter HW module for the sensor.

While this degree of outsourcing can be disconcerting for software engineers, it enables car manufacturers to build their complex products consisting of ~10.000 parts and ~80 ECUs without being world leader in all involved technologies – from mechanics over electronics to software. Just looking at the variety of software technologies implemented in cars – operating systems, safety-critical algorithms, mobile communication, … – it becomes apparent that it would be hardly possible to stay on top of all these technologies. The technological leadership in these individual fields is delegated to the suppliers – which act as a shared resource among OEMs. Relaying development to a market of suppliers reduces risk (through redundancy) and cost (through competition). In addition to the technologies developed by suppliers, car makers identify core features that they develop in-house.

For many years, the outsourcing model worked well for car companies. Just recently paradigms changed, triggered by U.S.-based tech giants entering the automotive industry with maps, app platforms (Apple CarPlay and Android Auto) and – most recently – self-driving car tech. It became apparent that there is, in fact, money to be made with software for cars, particularly with the data that flows through the software. At the same time, the complexity of software in cars exploded and demands tighter integration and more internal software competence. Consequently, we are now experiencing a steep rise in the effort to increase software competence and of in-house development; although this trend is still primarily focused on business-critical domains like autonomous driving.

When working on self-driving car software, the described segregation means that no matter whether you work for an OEM, a supplier, a software company or a startup, you’ll have a hard time getting access to the whole tech stack that runs a car, particularly the source code. You’ll likely be confronted black box binaries, you might not get access to raw sensor data, you might have to install a drive-by-wire system because you have no means to access the existing one. The automotive supplier landscape is heavily fragmented and you’ll have to talk with multiple parties.


As described above, it is a common pattern in the automotive industry, that a complex hardware-software product is built by a global network of suppliers, engineering service providers and the OEM itself on a strict time plan. To provide product updates on a regular basis it is necessary to align all partners.

To make progress while parts of the overall system are still in development, the automotive industry relies on prototype-grade hardware samples (A-sample -> B-sample -> C-sample) and software releases that increases in maturity until the actual production starts. This leads to intermediate states with semi-working prototypes of sensors and ECUs that can be painful to work with.

Even more important, it is hardly feasible to come up with a precise schedule for these nested worldwide activities; deadlines are missed, and you should be prepared for deliveries that are late and don’t arrive in the expected quality. Have a sufficient time buffer and a plan B ready to compensate for this.

How will “automotive software development” affect your work?

  • You’ll experience downtimes when sensors are broken, hardware was not delivered, or a firmware update breaks your development platform.
  • You’ll have to deal with half-working development cars, struggling to identify the bugs in your code within the quirks and random crashes of the system.
  • You’ll have to overcome outdated software engineering practices and deal with legacy code and black box software.
  • You’ll have to learn about automotive software engineering standards and comply with them.


Hopefully, this article was able to provide a realistic view of the competencies you should have to be a great self-driving car engineer, and on the specifics of the field compared to general software development.

If you are thinking of becoming a self-driving car engineer:
Even though every company has its own recruitment process and its own take on the qualifications required to get a position, the above-mentioned challenges and the importance of quality software engineering are universal.

In any case, as a great engineer, you should continuously strive to improve your skill set, and the links and concepts in this article should provide directions for your next steps!

Categories: Developmentgeneral

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.