Which programming language is used in the computer of a car?

After work today, I was reading about the throttle response adjustment procedure to mitigate turbo lag in the 3.5 EcoBoost F-150.  For the non car nerds, I’ll save you a click – it’s just a way to re-calibrate the electronically controlled engine throttle, making it feel more responsive when the gas pedal is pressed.  Nearly every part of a car is computer controlled or monitored these days, so I got to wondering…what computer language(s) do Ford and other manufacturers use to write the software that is used by the car’s computer?

Some quick Googling indicated that C is overwhelming used in car electronic control modules.  This makes sense as C is very common in embedded systems because it provides easy access to hardware, has low memory usage, and most importantly – it’s fast.  Auto manufacturers use a specific implementation of C known as MISRA-C (Motor Industry Software Reliability Association C).  MISRA-C is actually a set of guidelines for programming in C that helps avoid bad code which could cause dangerous behavior while a car is in operation.  I’m not a C programmer and a lot of this goes over my head, but MISRA-C is just a rigidly defined programming style. When implemented, it will ensure that common errors and pitfalls programmers can make will be avoided entirely when writing software for a car computer.

According to Wikipedia, MISRA-C was originally targeted at the automotive industry, but it has evolved as a widely accepted model for best practices by embedded systems developers in other industries including aerospace, telecom, defense, railway, and others.  Pretty cool stuff!  If you’re interested, here is some more reading on programming car computers:

https://www.quora.com/Which-programming-language-is-used-in-the-ECU-of-a-car

https://stackoverflow.com/questions/1044271/automobile-programming-languages

http://www.embedded.com/electronics-blogs/beginner-s-corner/4023981/Introduction-to-MISRA-C

http://www.eas.uccs.edu/~mwickert/ece5655/code/MISRA/MISRA_C_key_rules.pdf

Here is an interesting excerpt from the last link:

Rule 59 (required): The statement forming the body of an “if”, “else if”, “else”, “while”, “do … while”, or “for” statement shall always be enclosed in braces.  Basically, this says that from now you must clean up your act, you can’t write sloppy things like the else clause in following example:

if (x == 0) 
{ 
    y = 10; 
    z = 0; 
} 
else 
    y = 20;

The idea of this rule is to avoid a classical mistake. In the example below the line z = 1; was added. It looks as though it’s part of the else clause but it’s not! In fact, it is placed after the if statement altogether, which means that the assignment will always take place. If the original else clause would have contained the braces from the beginning this problem would never have occurred.

if (x == 0) 
{ 
    y = 10; 
    z = 0; 
} 
else 
    y = 20; 
    z = 1;

 

Advertisements

Excel is NOT a database!

Working in the GIS field, it’s very common to gather data for a project from various adhoc sources, and often that data is delivered in the form of an Excel spreadsheet.  I’ll preface by saying that there’s nothing wrong with delivering data in this way.  Excel is a powerful and widely used tool in the business world, and just about everybody who’s ever worked in an office has opened an Excel spreadsheet.  But too often in my GIS career I’ve come across projects that are using Excel spreadsheets as a master repository, storing large amounts of tabular data within.  It’s not hard to imagine how data integrity can quickly become compromised when people are copying/pasting/emailing/editing/saving spreadsheets, both on their local disk and shared network folders.

So why isn’t Excel a database?  It has rows and columns and you can sort data, create formulas, query data with features like VLOOKUP right?  While those things may be true, there are several reasons Excel is not a database:

  • Excel does not allow columns to contain only one specific primitive datatype – integers, floating point decimals, boolean, string, etc.  In a database table, each column or field can be constrained to only allow one datatype.  Often there will be a column you want to only contain a specific value, like an integer for an ID number.  Excel does not provide any data validation, and will not prevent users from entering an invalid value for for the ID.  A database table would throw an error, not allowing text to be used for the integer ID column.  A database table can also disallow null or empty values for a column, requiring every row to have a value, useful for ensuring each row has an ID number for example.  Excel does not provide this functionality.

  • Excel does not allow multiple users to open and edit the same Excel file.  This is a very limiting factor experienced by many users who share files on network drives.  I have personally run into this issue quite a few times, having to ask a somebody on the team to close a file that they had open so I could make some edits.  Inevitably, myself or somebody else would make a copy of the file to work on locally, and then we’d have issues reconciling the edits each person made with the master copy.  A database is designed to allow hundreds or thousands of users to concurrently query, view, and edit the data.

  • Excel is slooooow.  Like really slow.  The file size can balloon when there are many rows and columns, many formulas, and special formatting, filters, etc in the worksheets.  Sorting worksheets with many rows can be painfully slow, and  out of memory errors can happen when the content of the Excel file becomes too large.  Databases are designed to maximize performance and offer nearly limitless space, depending on hardware configuration of course. (SQL Server can handle databases of up to 524,272 terabytes!)  Excel files should be out of the question when data tables have hundreds of thousands of rows or more or data.

  • Excel does not allow you to query or join data from multiple tables.  Well, it does kinda…features like PivotTablesVLOOKUP,and HLOOKUP can provide functionality similar to some database queries by summarizing data, searching for matching criteria, etc.  I won’t get into how to use these features but they do not quite give the same functionality and flexibility as a database provides.  A significant limiting factor of Excel is not being able to link or join tables between files – all of these operations must be done with worksheets in the same Excel file.  Where Excel really falls short however, is one to many relationships.  This is an important concept in data relationships, and these kinds of table joins cannot easily be performed in Excel:

  • Last, but certainly not least, Excel doesn’t provide any data backup or recovery tools.  You may have some automatic file backups on your computer’s hard drive or on a shared network drive, but these may be insufficient if an Excel file is destroyed or altered.  A database will have built in features to backup data, and sometimes allow the data to be rolled back to a given point in time.  Bringing data back online after a hardware failure is generally an easy process with most database software.  A database can also keep logs of all changes to the tables, making it possible to undo a specific table update if an error was made.  On a related note, databases will offer options to securely access your data as well.  Excel files offer little in the way of protecting the data from unauthorized viewers.

To summarize, Excel is a great tool for data analysis, presentation, and mathematical and statistical calculations.  It is a not so great tool for storing large amounts of data.  It’s often difficult to maintain data integrity using Excel, especially among multiple users.  When the content of the file becomes too large, Excel performance drops significantly making viewing and editing the data difficult.

For any readers out there who currently find themselves in a situation where they feel Excel is not quite the right tool for data storage, I would urge you to explore using a database for your project.  There are quite a few options out there, both free and paid, for setting up a database.  It’s probably a little beyond the scope of this post to discuss all the various database storage and retrieval types and the specific database software, but since I’m a .NET developer I would suggest trying SQL Server Express.   It’s easy to set up a local database server and begin experimenting with migrating data from Excel into database tables.  You may also already have Microsoft Access installed on your machine as part of Microsoft Office.

The tokens that wouldn’t die

Here’s a funny post from The Daily WTF that shows some production code that allowed an API to issue tokens that were valid for nearly 50 years!

https://thedailywtf.com/articles/the-tokens-that-wouldn-t-die

I found this especially relevant to me as I’ve been working with API authentication recently and it reminds me to take extra care when dealing with theses security concerns, especially as someone who is new to this area of software development.

 

Making a POST request to an oAuth2 secured API using RestSharp

Recently, a coworker asked me how to best consume (using C#) an oAuth2 secured API which I had deployed.  I have been using RestSharp (along with JSON.NET) to make web requests in some of my applications recently, so I wrote a quick sample application for him demonstrating how to communicate with my API using those libraries.  I included it with the documentation for that API, but I want to share the basic concepts here as well.  Since the API is using oAuth2, the first step is to get an access token using an API key and password:

var url = "https://my.api.endpoint/GetToken";
var apiKey = "api_key";
var apiPassword = "api_password";

//create RestSharp client and POST request object
var client = new RestClient(url);
var request = new RestRequest(Method.POST);

//add GetToken() API method parameters
request.Parameters.Clear();
request.AddParameter("grant_type", "password");
request.AddParameter("username", apiKey);
request.AddParameter("password", apiPassword);

//make the API request and get the response
IRestResponse response = client.Execute<AccessToken>(request);

//return an AccessToken
return JsonConvert.DeserializeObject<AccessToken>(response.Content);

If you were successfully able to authenticate using your API credentials, you should receive a response that contains an access token and other information. Depending on the API you’re accessing, it may look similar to this:

{
  "access_token": "v5s5UckbViR9gZUXiu...",
  "token_type": "bearer",
  "expires_in": 43199,
  "userName": "api_key",
  ".issued": "Sun, 30 Jul 2017 17:05:37 GMT",
  ".expires": "Mon, 31 Jul 2017 05:05:37 GMT"
}

Now that the application has been authenticated and has be granted an access token, we can then provide this token when calling various API methods to get authorization.  Here is a sample POST request to my API, calling the DoStuff() method and including an object which contains the input parameters:

var url = "https://my.api.endpoint/DoStuff";

//create RestSharp client and POST request object
var client = new RestClient(url);
var request = new RestRequest(Method.POST);

//request headers
request.RequestFormat = DataFormat.Json;
request.AddHeader("Content-Type", "application/json");

//object containing input parameter data for DoStuff() API method
var apiInput = new { name = "Matt", age= 34 };

//add parameters and token to request
request.Parameters.Clear();
request.AddParameter("application/json", JsonConvert.SerializeObject(apiInput), ParameterType.RequestBody);
request.AddParameter("Authorization", "Bearer " + access_token, ParameterType.HttpHeader);

//make the API request and get a response
IRestResponse response = client.Execute<ApiResponse>(request);

//ApiResponse is a class to model the data we want from the API response
ApiResponse apiResponse = new ApiResponse(JsonConvert.DeserializeObject<ApiResponse>(response.Content));

And that’s pretty much it – the ApiResponse object now has all the data we need from the server response, whatever that may be depending on the API.  As you can see, both of these libraries together make sending and receiving data to/from a server very easy with just a few lines of code.  Getting authenticated with the API server, sending some data, and receiving a deserialized response is very simple.  More information about RestSharp and JSON.NET can be found here:

http://restsharp.org/

http://www.newtonsoft.com/json

 

Browser diversity is good for the web

Develop for the web, not for a browser! Here’s a good quick read about why having a single web browser for all operating systems and devices would not be a good thing:

medium.com/samsung-internet-dev/because-browser-diversity-is-good-for-the-web-910d1cbcdf3b

The article also included this quote from Steve Jobs that I found kind of interesting:

“What’s the point of focusing on making the product even better when the only company you can take business from is yourself?”

Competing browsers are part of what drives innovation on the web. Developing for the ever-changing hardware and software landscape of the web is definitely a challenge. It is important to test websites thoroughly – you want your users to have a good experience regardless of the device and browser they are using.