Encodings are important

I work as a hardware test engineer for a large computer hardware company. The principal software products that I work on orchestrate and execute hardware tests based on the product that is connected to the testing station. We mostly test units that are running Windows operating systems and learning batch/powershell scripting has been very useful in gluing test executions together. However, I recently discovered a few caveats in our process and this post will detail how I worked with and around the issues that I found in our polling and writing methods.

The first time I encountered the quirks in our resident polling method was when I was working on a test to make sure scanners would scan certain bar-codes properly. The test went smoothly but the result we were getting back was a garbled mess and so the test orchestration failed. I had remembered that there was a second polling method specifically calling out ASCII as it’s character encoding so I tried to use that one instead. After I switched polling methods the new scanner test was passing, however other scanners were now failing. I ended up sticking the decision to use ASCII or Unicode (our encoding of choice) into a configuration file that was then parsed in based on the scanner being tested.

The second time I encountered this particular quirk was when I was developing a firmware update utility. Initially, I was trying to run the encryption commands in a batch file, but piping it to a file always produced the wrong character encoding since batch takes whatever encoding the program uses and puts it out to a text file. So, instead of using batch, I wrote a powershell script that used the Out-File cmdlet to put out the results of the firmware update with the specified encoding. That way I didn’t have to modify the source code that orchestrates the test and reads files in.

The moral of the story is that whenever you are working with text files you should always make sure you know what encoding you’re expecting. Otherwise, you might end up with some strange characters in your debug session.