DJANGO APPLICATION FOR GEOSPATIAL DATA COLLECTION

 Introduction

 

This paper presents a web application that was developed in the Python programming language using the Django web framework and the Django geographic module GeoDjango. The "nnm" application is a web application that can be used in the electricity distribution company to collect the necessary data about the low-voltage network.

This paper is divided into three parts. The first part will describe the technology used in the development of the application, followed by a description of the database itself, Admin, Frontend, and finally a conclusion describing the application, the possibilities of embedding.

HTML

HTML (HyperText Markup Language) is the code that web browsers, such as Chrome, Firefox or Safari, use to display a website or application to users.

HTML stands for HyperText Markup Language. HyperText means that it is a type of text that supports hyperlinks between pages. Markup means we've taken a document and marked it up with code to, in this case, tell the browser how to interpret the page.

CSS (Cascading Style Sheets) is a language used to describe the layout and formatting of a web application written in a markup language such as HTML. CSS is the foundational technology of the World Wide Web, along with HTML and JavaScript. CSS is designed to allow separation of presentation and content, including layout, colors, and fonts. CSS is a completely independent language, separate from HTML, but practically cannot do without it. It can be said that HTML and CSS cannot do without each other. While HTML document structures create page elements, CSS styles and edits these same elements to give them the desired shape and function. CSS design can often be said to be an application. That is not completely true, it is not far from the truth. Without CSS, modern websites would look very plain and could not be used adequately.

 

JavaScript

JavaScript is a scripting programming language for dynamically programming web pages to make them as interactive as possible. It is also executed in the Internet browser of the user browsing the pages. Its syntax resembles the programming language C and Java, but it is considered a functional programming language.

JavaScript allows direct manipulation of web page elements, thus making them more interactive. Nowadays, JavaScript is more and more important because there are a large number of existing libraries that are accessed with that language. JavaScript libraries facilitate the work itself and speed up programming due to many ready-made functions that are often used in the development of web applications.

 

PostgreSQL

PostgreSQL is an open-source object-relational database system. It uses SQL as a language for capturing data, and can also use most scripting programming languages such as Perl, Python, and Ruby. The already built-in language is PL/pgSQL which is similar to the procedural language PL/SQL used in Oracle databases.

It contains almost all the possibilities from the SQL:2008 standard and is therefore the most complete open-source relational database. PostgreSQL does not include support for managing spatial data, but the PostGIS add-on adds that capability.

PostGIS

PostGIS is a plugin that adds the ability to manage spatial data to the PostgreSQL database and can then be considered a spatial database. PostGIS functions comply with the OpenGIS specification "Simple Features Specification for SQL".

When PostGIS is installed in the PostgreSQL database tables, a "geometry" type field can be added, which can contain points, lines, polygons, or a collection of the specified objects. In order for it all to make sense, it is necessary to use PostGIS functions to manage that data. This is achieved by SQL queries that contain functions for manipulating geometric fields. There are several types of such functions such as:

- geometry constructors (for creating geometric fields),

- functions for retrieving properties or individual elements of geometric fields,

- functions for editing geometric fields,

- functions for converting geometric fields into some of the output formats (WKB, WKT,

GML, GeoJSON, KML, SVG, GeoHash) and

- Functions for measurement and relationships between geometric fields.

 

Python

The Python standard library is well adapted to writing web applications, but it does not include modules for direct communication with databases, but provides an API (Application programming interface) with which such modules can be created. Web applications most often use databases, so such modules outside the standard library are already available for all important databases (Oracle, MSSQL, PostgreSQL, MySQL).

One such module is psycopg2 for communication with the PostgreSQL database. It uses all the capabilities of the Python DB API and can be safely used in a multi-tree environment (trees can share the same connection to the database).

Python is a powerful, flexible programming language that can be used to develop web applications, create games, and more. Python is a high-level language that is very easy to read and write.

Python is also an interpreted language - which means that it is interpreted when the source "code" is directly executed with the help of an interpreter without the need to compile or translate it into an executable form.

Python is an Object Oriented language, users are allowed to manipulate data structures called objects, in order to write a program and execute a command at the end. The Python programming language was created in the early nineties of the last century. Its author is Guido van Rossum. He was employed at the Stiching Mathematish Centrum (CWI) in the Netherlands at the time of the language's creation.

Given the very name of the Python language, the author let us know that programming can be as fun as the series of the same name, after which the program was named "Monty Python's Flying Circus", or in English "Monty Python's Flying Circus".

Linux is the right environment for Python, however it can also be used on a Windows system, but its environment is simply made to adapt to Linux operating systems, which is good, considering the ways of programming and the speed of command execution.

 

Gdal

GDAL (Geospatial Data Abstraction Library) needs to be installed $ sudo apt install gdal-bin

 

GeoDjango

GeoDjango is a built-in application that is included as a contrib module in Django. It is actually a complete framework that can be used separately from Django. It provides a set of tools for building GIS web applications. GeoDjango also provides Python bindings for popular spatial libraries such as geos, gdal and can be used separately without Django in any Python application or directly in the shell. GeoDjango integrates very well with the Django framework and provides a set of geometry fields defined by the Open Geospatial Consortium (OGS), which can be used to map different types of geometries in geospatial databases:

- GeometryField, is the base class for all geometry fields in GeoDjang,

- PointField, is used to store GEOS point objects,

- PolygonField, is used to store GEOS polygon objects.

To be able to work with GeoDjang, two things are needed:

- spatial database and - geospatial library.

A spatial database is a database that is optimized for storing and querying data representing objects defined in geometric space. In order to fully enable all functions of GeoDjang, the following geospatial open source libraries were installed; geos, gdal, proj.4, psycopg2.

GeoDjango is a plugin for the Django environment that adds the ability to manage spatial data. The development of GeoDjang began in the fall of 2006 independently of Django, and in 2008 it became part of the Django environment. The GeoDjango plugin enables quick and easy creation of web applications that use spatial data. In that case, a spatial database must be used, and GeoDjango supports the following:

- PostgreSQL with PostGIS plugin

- MySQL

- Oracle

- SQLite with SpatialLite plugin

To create a GeoDjango application, it is necessary to choose the operating system on which the server software will be executed. This paper uses the Ubuntu distribution of the GNU/Linux operating system version 22.04.

The PostgreSQL spatial database with Postgis and psycopg2 extension was used during the creation of this project. The tool used Visual Studio Code as the development environment. Also, when creating this project, web technologies html, css and JavaScript were used.

The first version of the Ubuntu distribution was created in 2004, based on the Debian distribution. With

RedHat and Debian, the Ubuntu distribution is often used for web application use due to its simple operating system and easy installation and configuration of server software.

In a test environment, a Django application can run its test web software, and as a database it can use an SQlite database consisting of a file that the Django application accesses using the built-in function library. such a combination is not suitable for a production environment due to the poor performance of the embedded web software and the lack of spatial data management functions in the SQLite database. For the production environment, it is necessary to choose an independent web server and spatial database. The web server used in this work is Apache with the mod_wsgi plugin, and the spatial database is PostgreSQL with the PostGIS plugin.

 

Ubuntu is a distribution of the GNU/Linux operating system. Based on the Debian distribution,

Ubuntu pays the most attention to ease of use, freedom from all restrictions, regular release of new releases (every 6 months) and ease of installation. Ubuntu is sponsored by South African businessman Mark Shuttleworth's private enterprise, Canonical. The name of the distribution comes from the African ideology of ubuntu (roughly: "humanity towards others").

 

Django

Django is a set of tools and a foundation for creating web applications in the Python programming language. He is alone

Django is written in Python and was created to speed up and simplify the programming of web applications as much as possible. It contains ready-made tools for standard development procedures such as:

- object-relational data models, - automatic browser for administration, - elegant URL display.

 

Web application programming often involves repeating the same tasks over and over again. Such tasks are:

- creating an administrative browser,

- user authentication,

- user rights,

- creation of data entry forms, - validation of entered data and - internationalization of web applications.

 

In other programming languages and environments, such tasks are painstaking work. U

In the Django environment, the listed procedures are simplified and boil down to writing a few lines of Python code. The object-relational data model avoids writing the SQL queries that Django automatically generates.

With more complex web applications, it is necessary to isolate the logic of the application from the display part of the application. Django uses the MVC (model-view-controller) architecture where the model represents the data model in the database, the view represents the display part of the application (what the end user sees), while the controller is responsible for the logic of the application itself and connecting the model and reviews.

Django uses different names for MVC elements, which is a bit confusing. The Django view actually represents the controller, and the template represents the view from MVC, so the implementation of MVC in Django can be called MTV (model-template-view). When a user types the address of a Django application into their Internet browser, that address is first accepted by the URL forwarding module. This module consists of regular expressions according to which the view to be activated is selected. The view communicates with models that are responsible for communicating with the database. The model returns data from the database to the view, which is then sent to the template (usually HTML). The template generates HTML code according to the received data and sends it back to the user. If cache is used, or if data from the database is not needed, some of these steps can be bypassed.

Application creation in Django and installation

The application was developed in the Visual Studio Code editor, primarily because of the many extensions it offers, but it is also possible to use other program editors such as Sublime Text or Notepad ++.

Creating the application starts with the creation of the project, the terminal offered by Visual Studio Code is used for commands:

This command creates a new directory where the manage.py and nnmreza files are created. The nnmreza file is the main folder whose files are treated as a Python package, and manage.py is the file used in working with the terminal, for starting the server, migrating data to the database.

To create an application within the nnmreza, which will be called "pillars", it is first necessary to position it within the project, i.e. nnmreze and then the startapp command.

Application setup

After the stubs application is created, it needs to be added to the list of installed applications in settings.py located in the main nnmrez directory. Settings.py is a core file or

"core" in the Django project. It contains everything needed to configure the project in creating the application, from database settings, logging records, where to find static files, API keys if working with an external api and many other settings.

Elements in the list of installed applications are written in quotes as a string and separated by a comma.

 

View.py or "View"

In the Django file view.py, functions are defined that retrieve data from the database, and then pass it to the forms that will be displayed to the user. For each form you need to write a function.

Template

A template or "view" represents a folder. A map named template is created in the application and an html file (map.html) is created in it. In that file, data is displayed depending on the functions we defined in view.py. Forms cannot use standard typing of commands in

Python, because they are still html files, but that's why Django has its own language that enables it.

Adding javaScript and CSS files

The next thing to do to create the application is to add the "static" folder. The folder can be created in the Visual Studio Code browser by selecting the icon or in the terminal with the command:

U kreirani folder „static“ koji se nalazi u nasoj aplikaciji stubovi kreiramo fajlove map.css i map.js

Adding a new URL

The next thing that is needed is to create a new url.py in the pillars application, which normally does not come with a formed url.py and needs to be created every time.

In urls.py it can be seen that a link was created from views.py with the created class "StuboviMapView" and based on it the path "map/" was created, which is entered in the browser.

Modification of urls.py in nnmreza

In the main nnmrez folder, the urls.py file comes with the admin path already created. For the needs of the application and its launch in the browser, in this example we add the include module and create the link "pillars" or we can write something else.

 

Creating a spatial database

To import a model created for a database, you need to create a database in postgresql. We can do this in two ways through the terminal in psql or by installing a gui for postgresql called pgAdmin4. In it, the base is created in an easier and faster way, and the gui allows visibility and easier manipulation.

After the database has been created, it is necessary to create a postgis extension that will allow us to store geometric data. If the extension is properly formed, the spatial_ref_sys table will be formed in the database.

Changing data in settings.py for base

After creating the database in postgresql and the postgis extension, it is necessary to modify the data in settings.py. Settings.py is located in the main nnmnet project.

By default, "ENGINE" comes with Django's sqlite3 and the spatialite extension, which we change to postgis, and we enter the data according to the principle that was created in the database. According to user experience, for geoprojects and web applications with a "geo" sign, it is recommended to use postgresql, especially because of the pgadmin it offers.

 

Importing GeoDjango

To activate GeoDjang and all its features, it is necessary to modify it

INSTALLED_APPS in nnmrezi/settings.py by adding django.contrib.gis

Model.py

In a Django application, a model represents a table in a database. Models in Django are defined as classes, and they represent data in the database, i.e. each class represents a table in the database itself.

Variables represent columns and are used to define relationships. The model for the database that we will create in postgresql:

The 'stub' class contains the attributes 'stubovi', 'lokacija', 'grad', 'konzola' and 'provodnik'. All fields except the location are text type and maximum length is 255 characters, while the location model stores points or "point". Point objects are instantiated using arguments representing the coordinates of the point. Django with its tool for automatic synchronization with the database creates a table in the database with the specified attributes, and this structure does not need to be adjusted additionally.

Data migration

After the model has been created and the database with the postgis extension has been set up and the ussettings.py connection has been set up, the data from the model must be migrated with the command

By default, 10 tables are created in the database after the command. In them we can see administrations, data for migrations, logging, etc.

The next command to create a new table in which we store data from the application is:

Admin.py

To import new pillars, it is necessary to add the pillar admin in the application stubovi/admin.py

GeoDjango contains add-ons for the administration panel that allow the creation and modification of geometric shapes. Basic is a GeoAdmin plugin that uses the background

OpenLayers maps. For more detailed backgrounds, OSMGeoAdmin can be used, which uses Open Street Map layers. The activation and setting itself can be seen in the attachment.

Login to Admin

To start the admin, you need to create users

When creating a superuser, it is necessary to enter information about the name, email, which is optional, and password. With these data, you log into the Django admin area. After creating the necessary data, we use the command to start the default server http://127.0.0.1:8000/. If our default server is busy, we can also use e.g. 9000, we just have to enter in the command which server we want.

After logging in, the Django admin area is displayed. In order to facilitate the manipulation of the model and to facilitate the users of the application, such as editors, administrators who usually do not have any programming experience, the practice of creating the administrative part of the application is often used.

This part serves to manage the application model through the user interface, without using code. In the example of this application, this would mean that in the pillars application, the administrator can log in to it, fill in data, add users, delete data and edit it. There is also a Django allauth module which is an integrated set of Django applications dealing with account authentication, registration.

The data entered in the columns application is recorded in the postgresql database

The data is displayed on the map with a marker defined by the leaflet, and the base layer or base is from OpenStreetmap, with a popup that calls the column name in the database.

This paper describes how to create a web application using Django with GeoDjang. It turned out that the application is quite flexible and that it is easy to add new features that have been omitted in this work due to the focus on Django's geo features. The possibilities that arise are the expansion of the "pillars" application, which should contain Bootstrap, a photo, more bases and data that needs to be entered. Additional classes in models to form additional tables of substations, their coordinates and attributive data with addresses, photos.

Potential users of this application are, in addition to distribution companies, cable operators who also use a network of towers for data transmission.