Developing software is a hard process. Developing embedded software is an even harder process. With traditional pitfalls of syntax error like missing colons, logical errors like setting a variable with the wrong value, developers are challenged by complex low-level hardware details. This means additional ways to break the software, from having assembly code not executing as expected to inadvertently writing to wrong memory location.
One of the key factors in software development is to reduce the number of variables that can break the codebase. And in order to achieve defect-free products, engineering teams should test software/code more often. It is much easier to debug and fix 100 lines of code(LOC) than 10,000 LOC. Earlier the developer identifies bug/defect in the development cycle, cheaper it becomes financially to fix them. One of the strategies used nowadays is continuous-integration and continuous-delivery i.e. CI-CD (more on this later). These new workflow helps to identify bugs in the early stage of development. This ultimately leads to a reduction in financial overheads as well as the total time to market is reduces as bugs/defects are fixed in the early stages of development.
Platforms available for embedded software development
When starting to develop embedded software, it’s rarely a good idea to start testing on the end target. Be it an ECU(Electronic Computing Unit) of a vehicle or customize hardware for consumer product. Lots of the time providing actual target devices to development or testing teams won’t be feasible. This may be due to the fact that the target device is in the design phase or not a cost-effective solution due to the large team size. Engineers face a lot of issues while developing an embedded system. For example issues with the physical connection, connection protocol, hardware driver, architectural features, toolchain settings, and the source code – just to name a few are all possible and developing software with these many variables in the start is not a sound strategy.
Many times to cut the overall cost of system development, organizations try to take an easy approach which in short terms seems effective but the long term consequences are high. Yes, you guessed it right. Using native compiler ( i.e. x86 host compiler) to test the embedded code. Due to differences in the architecture of end product and x86 architecture, majority software may not be compatible for host compiler. Which eventually leads to compiler, linker or runtime errors(more on this later). Setting up a test environment may become a huge task in itself.
There are many options to develop (and test) embedded software on, and the most appropriate platform depends on the project priorities and how far along with the hardware and software development. There are two main platforms: hardware-based and software-based.
Hardware Platforms
Hardware platforms include development or off-the-shelf boards, FPGAs, etc. Testing on such hardware platforms represents the final product. These are very good options for quick prototyping kind of work. In general, these hardware platforms are good for accurate software testing as they mimic the final product’s system architecture but are difficult to scale to a large number of tests and bigger teams due to expensive cost and maintenance. They are traditionally used for a smaller number of more encompassing tests (such as system tests) and for software development towards the middle and end of a project’s lifecycle. Having said that depending upon the domain in which one is serving, there are industry-specific guidelines where organizations need to choose the actual hardware approach to test their code. One such example would be the aero industry due to the obvious safety-critical nature of systems.
Software Platforms
Software platforms include native executable, instruction set simulators, and virtual models. The native executable is cross-compiled to run natively on an x86 based machine with no virtual environment necessary. Instruction set simulators are used to run code compiled to run on the final target architecture. They are run in a virtual environment that simulates the instructions of the final target architecture. These platforms are useful at the start of a project to ensure that basic functions and code flow are working as expected and are quite simple to scale to a large number of engineers. However, they become less effective as software begins to interact with the details of the final target architecture and various peripherals. When software testing needs to become more accurate and representative of the final target system then having actual hardware is the only option.
Let's try to understand which platforms can help reveal the types of software bugs that are very helpful. Below is a list of each software development platform, with examples of what types of software errors the platform catches. Note that as this is not an exhaustive list and is intended to convey a general idea only and depending upon the testing phase i.e. unit testing, integration testing, system testing few options may or may not be feasible.
Native Executable
- Syntax errors
- Program flow errors
Instruction Set Simulators
- Syntax errors
- Program flow errors
- Toolchain issues (though sometimes tied to target CPU behavior this may not be caught)
- Implementation defined behavior (not all)
Development board
- Syntax errors
- Program flow errors
- Toolchain setup and optimization errors
- Floating point and subnormal number inaccuracies
- Implementation defined behavior
- Coherency maintenance
- Cache maintenance
- Program timing issues
FPGA
- Syntax errors
- Program flow errors
- Toolchain setup and optimization errors
- Floating point and subnormal number inaccuracies
- Implementation defined behavior
- Coherency maintenance
- Cache maintenance
- Program timing issues
- Some third-party driver errors
- Code depending on third-party IP errors
End Target
- Syntax errors
- Program flow errors
- Toolchain setup and optimization errors
- Floating point and subnormal number inaccuracies
- Implementation defined behavior
- Coherency maintenance
- Cache maintenance
- Program timing issues
- Some third-party driver errors
- Code depending on third-party IP errors
- Remaining peripheral driver errors
- Hardware-software expectation mismatches
- System requirement mismatches
Conclusion
From the above list, we now know the target is more valuable and catches maximum issue but due to stated reasons at the start of this article, organizations opt to test using native compiler i.e. on the host(x86). Apart from missing to catch the above-mentioned issues, testing on the host not only challenging but also error-prone and end result may not be accurate. Using a host-based compiler for embedded software development may not catch bugs in the earlier stage of the development cycle. In such cases, we can make some tradeoffs owing to financial overheads by selecting a software-based solution by incorporating at least Instruction Set Simulator in software testing. Setting up an instruction set simulator is not difficult and most of the leading compiler IDE comes preinstalled with simulator debugger support. Apart from compiler vendors, there are few 3rd party tool vendors who have support for Instruction Set Simulator which is useful in cases where the cross compiler does not have inbuilt simulator support.
Comments