How to present 3000 records a day in an accessible way
It is assumed that 65% of the population are visual learners. But for even more people, visually accessible data is very easy to read. If it is necessary to process a huge amount of data, such as the 3,000 records a day that our client the City Water and Drainage company in Wroclaw (MPWiK – Miejskie Przedsiębiorstwo Wodociągów i Kanalizacji) in Wrocław collects day in the Mokry Dwór Research Station, the appropriate presentation of information is essential. See how we increased efficiency and improved the work of one team of a major water and sewage company.
MPWiK Wrocław approached us with a request to create an application that allows for clear graphical visualization of the measurement values of water parameters taken from subsequent stages of the water treatment process in Mokry Dwór Research Station. This data was collected through the existing SCADA system. It is true that the client had charts, but they were not sufficiently accessible to the employees. It was also challenging and time-consuming comparing different types of data – based on the original charts, it was very hard to get the full picture. This was especially true when the values of each parameter on a monthly scale were presented on 30 separate charts – one for each day.
Our project was characterized by many requirements and tasks for Stermedia programmers. Thanks to our use of SCRUM methodology throughout our work, we successfully completed the project on time.
MPWiK Wrocław wanted to:
- be able to interactively change parameters and data range
- have access to data in the form of tables and charts
- be able to compare data in a chosen time interval and units
- present measurements for the same parameter (with the same unit) from different sensors within one graph
- use an understandable legend
- use the application on various devices (web and mobile)
- be able to generate .CSV files
- be able to assign employees personalized accounts protected with login and password
The client gave us a description of the data series that needed to be displayed. It had specific needs, for which we selected proven methods, such as ready-made frameworks and libraries. This allows you to save time and budget. We can focus on the right job. The project was broken into two microservices; a frontend – written in Angular 9; and an API backend – written in Python using the Django framework. This communicated with the Microsoft SQL database in order to obtain data in real time.
We successfully implemented both microservices at the same time. The front-end part was developed on the basis of sample data, and only later was combined with the back-end. As part of the work on the backend, we created the environment and set up its skeleton using the aforementioned frameworks. The next step was to solve the issue of transferring data to the MSSQL database and configure the appropriate endpoints to download the appropriate data.
FACING MAPPING ERRORS
We faced the challenge of working with the database (backend) and mapping data series (frontend). The data was stored in the database where the rows corresponded to the subsequent measurements, and the columns corresponded to the data series. The problem was that several sensors had changed their position, but the names of the columns in which their readings were stored had not been updated. With a large overall number of sensors, this caused several mapping errors. Finally, thanks to our very good contact with the client and their domain knowledge, we correctly verified all endpoints. Daily, short team meetings, sometimes attended by the client, were useful in this regard. Thanks to this work system, each party was aware of the current arrangements and could freely exchange knowledge.
The new look of the system for Mokry Dwór Research Station in the City Water and Drainage company in Wroclaw means better quality for many employees. It is a joy to participate in a project that has helped many people.
As a result of our work in the SCRUM methodology, we provided an application that greatly facilitates the interpretation of data. It allows the user to freely choose their scope and type. In addition, interactive charts allow for the checking of the value at any point and zooming in on the interesting fragment on both the OX and OY axis.
The tool created by the Stermedia team also allows the presentation of data in the form of tables and their export and download as CSV files. Thanks to the Responsive Web Design approach, the application is suitable for use on desktops and laptops, as well as on mobile devices.
Our agile approach to the work makes cooperation with the client very good, and our product 100% met the initial assumptions. The amount of time needed for data verification has significantly decreased and new possibilities of comparing data have opened up thanks to implemented improvement.
Frontend – Angular 9
Backend – Python API, Django framework, Microsoft SQL database
Let's talk about your project
Leave us a message so we can understand your idea and goal.
In the next step, we will focus on requirements and provide a quotation.