Weapon of Choice

While working at REDACTED, I reported the following issue.

• Back end programmers tend to provide raw data only.
• Front end programmers can retrieve data from many endpoints.

Shouldn’t we then use the browser to aggregate data?

No, because if there is a reason for having interfaces between different systems it is to hide their inner workings and offer more coherent, more abstract, more complete and, in general, easier to understand operations and results.

The Front end side needs both aggregated and raw data fron the Back end.

For example, take a search results page for authors’ profiles based on their language.

1. A language select box needs to show English, Spanish, … (raw data) for the user to search one
2. The resulting profiles need each to show the number of authored documents, and their titles (populated documents)

Of course I could use three normalized endpoints:

1. getLanguages() for the options in 1.
2. getProfiles(page) for the results in 2.
3. getDocuments(authors) for the titles in 2.

but that would

• add a call from Front end to Back end, causing unnecessary slowness for the end user;
• add knowledge to Front end about Back end implementation details;
• add complications to Front end, just because.

Those complications are:

1. get the authors from the profiles in the current page;
2. asynchronously get the documents from getDocuments(authors), taking care of possible errors;
3. then group the titles and the count of those documents by author;
4. then make those additional data appear in matching rows on the page.

Could the operations above be performed in the Back end? Not only they could, but they would be easier and faster there, because getDocuments would not be asynchronous.

Instead, a Back end engineer told me he wouldn’t populate a response because it would set a precedent and all other Front end engineers would ask the same at any later time. Another Back end engineer said that long ago they had foreseen the need for a data aggregator but lacked will power to actually program it. (I couldn’t believe these guys!!)

Data population is so common a concept that many Back end APIs provide ways to get only selected properties. For example, there could be a better endpoint like this:

getProfiles(page, { 
    populate: ['documents', { 
        populate: ['title'] 
    }]
});

While working at REDACTED, I reported the following issue.

The Back end team routinely changes the services they offer to the Front end team. Changes occur both at service level (some services are added, others are removed) and data level (some attributes are added, others are removed). This is inevitable during development, but it doesn’t need to cause random bugs.

Currently, those changes get lost because the Swagger pages (which is the tool they use at REDACTED to publish HTTP endpoints) are published anew each time they want (which can be many times a day), and this tool does not show any differences with respect to a previous version.

Soon after detecting this problem, I wrote a small script to access the Swagger pages and generate the corresponding data interface (many thousands of lines of attributes with their names and types, readily usable from TypeScript). This allowed me to easily compare snapshots of the Back end services taken at two different times, and find out whether, where, and what changes occurred in between.

But that was just a partial solution to a process problem. In fact, not only those changes are still lost today (they are not using my script) but the Front end side still interfaces with the Back end side by means of a collection of classes translated from the Swagger pages into TypeScript: for no reason at all (cheap laborers?) they do it manually even though it’s a mechanical operation.

Notice that additional improvements would be just around the corner when this process will be automated. For example, if a Java program generated the TypeScript services at each new version, not only the changes would be easy to spot (with a simple text comparison tool), but we could also collectively push these changes to a repository as a merge request, giving the Front end team time to evaluate the impact of those changes and decide how to deal with them.

While working at REDACTED, I reported the following issue.

Any engineer is a world apart at REDACTED, so given that programmers work alone on a big chunk of functionality during a few weeks in a row, they produce lots of code which is never harmonized with the rest of existing functionalities.

There are many reasons. Clearly there is little to no interest in improving code quality at a management level, because they think that technical debt can always be fixed later by adding more cheap laborers, when it’s probably the exact opposite of that, i.e. you can always add more cheap laborers to produce more code if the technical debt is kept at bay along time.

Another reason (in Front end) is that the size and number of code changes in a single merge request makes it nearly impossible to perform meaningful code reviews, and in fact seasoned programmers skim through them very fast and accept true atrocities.

For example, in the same API

• they allowed the same properties to have different names, having to use one name or the other depending on the state of the object, like
  • activeGroup.name
  • pendingGroup.groupName
• they allowed the same concepts to have different values, having to use one value or the other depending on the endpoint you use, like
  • operations = ['CREATE', 'UPDATE', 'DELETE'];
  • operations = ['C', 'U', 'D']
• they allowed the same operations to have different results’ structures, like
  • `POST {name: ‘John’, age: 32}
    • success response: {name: ‘John’, age: 32, created: …}
    • error response: {error: ‘Age should be less than 30’}`
  • `POST {city: ‘Barcelona’, temperature: ’18 C’}
    • success response: ‘OK’
    • error response: {msg: ‘Italian cities only, please.’}`

Of course there are many more typologies of incoherences in our code base.

It could seem that they are just little inconveniences but they contribute lots of lines of code to the app. For example, a thing so simple like comparing two instances of the same document in different states, becomes a mess (i.e. more intricacies and bugs) when the names of the properties change according to the state.