Month: April 2023

Tracking system integration with git repository

Posted on by Alyona

Developer's workflow consists of many different tasks and projects. Tracking systems are used to structure them and organize convenient work. Small projects use Excel spreadsheets quite successfully for this purpose. However, project development requires more convenient solution implementation.

A tracking system is a software product, project management system that ensures the fulfillment of tasks, workflow planning, monitoring processes and results. This tool is useful for all team members: developers, project managers, team leads, top management, etc.

Now, the market offers large number of options for tracking systems: Jira, Mantis, Trello, Redmine, PivotalTracker, Bugzilla, Commind, etc. One of the most popular solutions is Jira.

Jira is a comprehensive solution from the Australian company Atlassian, which includes Jira WM (for working with business processes), Jira SM (for service disk building), Jira Software (for software development projects). These products are grouped into the Jira Family Product.

This tool is a paid system with an interactive dashboard for monitoring tasks movement and controlling their progress in a specific project. It is also a bug tracking and a convenient tool for project management, especially for agile teams. The main Jira goal is to simplify workflow management. The system has a wide functionality with the ability to supplement it with the help of plugins. There is a trial version of this tool for 7 days, which is available on the developer's official website after user registration.

Jira benefits:

  • Wide functionality (the ability to adapt the tool to projects of any complexity);
  • Great potential for integration (ability to integrate with various services such as GitHub, Salesforce, Outlook, Slack, Gmail, Teams);
  • Possibility to expand the functionality using plugins (Tempo, Script Runner, EazyBI, Big Picture, Structure);
  • Ability to work with Scrum and Kanban methods.

It is worth noting the rather long tool setting process for a specific project, workflows and a complex interface. The reason for this is the wide functionality of the system. However, tool study and practice allow to fine-tune it and optimize it during use.

Jira features depending on teams, roles and purposes of use:

  • Project Manager (PM): a great assistant for the project manager However, a complex and multifaceted tool requires detailed study to effectively use all its functions. Using tool capabilities, PM get a possibility to propose interesting initiatives for process optimization. Official Jira certification increases the competitiveness of a specialist;
  • For project management teams: the ability to use a ready-made template that the tool offers or create own process with the ability to track all tasks, their way through all process stage. The administrator has the ability to configure access with different rights for different users. Also, the tool allows to create detailed reports if there is information about the project;
  • For software development teams: the availability of planning tools allows members to manage budgets and requirements. Jira integrates with various CI/CD tools. This contributes to transparent management of the entire software development cycle. Gradual and safe introduction of new functionality is ensured by built-in features enabling and disabling features;
  • For DevOps teams: Atlassian's open, integrated toolset is based on Jira Software. The tool integrates with native and third-party tools, as well as code management solutions (Bitbucket, GitHub, GitLab), documentation and knowledge management tools (Confluence), monitoring and management tools, etc. This tool flexibility accelerates a quality software delivery;
  • For Product Management Teams: Teams have the ability to create a roadmap for all projects. This allows to make long-term plans, track progress and share information;
  • For bug tracking: tasks (bugs/problems that occur during software development) can be seen in the backlog and prioritized. Automatic issues prioritization is provided by Jira's workflow management system;
  • To manage tasks: create tasks for all team members, add detailed information, due dates and reminders. It is possible to divide large tasks into several small ones, create subtasks for certain team members. Other users can track tasks progress and receive a notification when it is completed. The current tasks status can be viewed using the dashboard.

The ability to integrate a tracking system and host a remote git repository is very convenient in managing Qlik projects. QOps can be used as a bridge between the source code in a repository and the final application. This kind of integration allows to automatically embed links to tasks in the tracking system into the source code of the application. As well as this feature allows management to track the progress of tickets and the corresponding source code migration between environments and/or versions.

Choosing a service provider for Qlik projects management

Posted on by Alyona

Git is a distributed file version control and collaboration system that was developed in 2005 by Linus Torvalds. The developer explains its name (git) with a bit of sarcasm: «I am a selfish scoundrel, and therefore I name all projects after myself. First Linux, now git.»

Now Git is the most popular and free tool that is a set of command line utilities. It allows to track the history of software development and work with entire teams from anywhere in the world on one project. Each change is added in the form of commits that allows to track, revert the change if it’s necessary and revert to previous versions.

In addition to being comfortable, flexible, and able to maintain development history, using Git greatly reduces development errors and data loss. Similar SCM version control systems are Mercurial, Subversion, Darks, Bazaar. However, Git has some advantages:

  • high speed and transparency;
  • ease of use in non-linear development;
  • efficiency for projects of any scale, including large systems with 1000 participants;
  • saving snapshots of repositories that ensures high speed;
  • the ability to restore the code from a local copy in case the server with the remote repository fails;
  • the ability to quickly download code from a remote server if the local copy is damaged.

Git main tasks:

  • saving code and changes history;
  • saving information about the user who makes changes to the code;
  • recovery of any code version;
  • consolidation of different versions, version changes;
  • preparing code for release.

The use of Git is carried out with the help of special hosting and repositories.

GitHub was created in 2008 and bought out by Microsoft for $7.5 billion in 2018. GitHub is a source code hosting site and a large social network for developers with 20 million users who can view each other's code, help to develop, and leave comments, and 80 million repositories worldwide. Users can create their own repository and publish their work. Free use is possible only for public open-source projects. The platform is written in Ruby on Rails and has a large number of open-source projects available to the public.

GitHub benefits:

  • free service;
  • high search speed in the structure of repositories;
  • a large number of users, which ensures a quick search for help in solving a problem;
  • availability of practical tools for collaboration and integration with Git;
  • easy process of integration with other services;
  • the ability to work on TFS, HG, SVN.

GitLab is an alternative to GitHub and is a web repository that provides free public and private repositories. GitLab was developed by 2 Ukrainians: Dmitry Zaporozhets and Valery Sizov, using Ruby and some parts of Go. The architecture was later enhanced with Go, Vue.js and Ruby on Rails.

GitLab is a complete DevOps platform for project planning, source code management, monitoring, security, and more. It also offers wiki hosting and a bug tracking system. Using GitLab significantly shortens the product life cycle and increases productivity. This, in turn, adds value to the customer.

Gitlab benefits:

  • ease of setting;
  • user-friendly interface and tools;
  • unlimited number of free private storages;
  • the ability to integrate a large number of APIs and third-party services;
  • reliability and non-failure operation;
  • the possibility of self-hosting;
  • open-source license.

BitBucket is an analogue of GitHub that is designed for hosting projects and their joint development. The service was developed by the Australian company Atlassian, which created Jira and Confluence. It allows to create public and private repositories based on Git technologies for free, has the ability to integrate with Jira and Snyk, and built-in CI/CD capabilities. The service is a great solution for small teams.

BitBucket benefits:

  • unlimited number of open and private repositories;
  • support for public and private Git repositories;
  • possibility of integration with other solutions (Jira, Trello);
  • the ability to integrate with third-party services (Twitter, Google Analytics);
  • the possibility of using the service for more than a million teams with 10 million developers around the world.

Jenkins is an open-source system based on Java. It provides automation of software development process parts without involving people and the process of continuous software integration. Jenkins is widely used in companies where there is a need for automatic applications deployment. The system is free, runs on Windows, macOS and other Unix-like operating systems, as well as has integration with Docker and Kubernetes. Jenkins does not contain a repository with source code, but connects to existing ones via web hooks. Thus is a universal tool for CI/CD regardless of the chosen hosting of remote repositories.

Jenkins benefits:

  • free use;
  • wide possibilities of integration, large plug-ins library;
  • large and active community;
  • distribution of load on different machines;
  • great option for large projects.

QOps can work effectively with all the above services.

GitHub and GitLab include the ability to install runners that will later execute commands from a .yml file included in the source code repository when a certain event is reached. Usually such an event is sending source codes (push) or merging branches (merge) in a remote repository. At the same time, the syntax for compiling .yml files is slightly different, although it describes the same essence of runner behavior and managing the process of application building, testing and subsequent deployment. Both hosting systems allow to install their runners in a Windows environment that is not yet available for BitBucket hosting.

Below is QOps usage with the above systems. For GitHub, a .yml file is a structured steps set that consists of successive steps. The example below describes 3 steps – init-build-reload.

Successfully executes pipeline looks like this.

For GitLab, the .yml file is the same structured steps set, the differences are a different name for the keywords and more flexible work with variables. Unfortunately, developers don’t use a unified .yml file format and there is no interoperability between them.

Upon successful completion, a pipeline view consisting of the same stages is presented below. It is worth noting here that GitLab has more developed interactivity. For example, in the pipeline, it’s possible to set up manual confirmation from the user.

QOps currently works exclusively in the Windows environment. If BitBucket is chosen for source code hosting, the automatic pipeline can be developed using the Windows version of Jenkins. The versatility of the latter and plugins variety allows to link any remote repository through web hooks. The pipeline structure is written as a JSON object, if configured through the interface of Jenkins itself. And for our example it contains all the same 3 stages – Configuration-Build-Reload.

The result of successful pipeline execution in Jenkins is shown below. At the same time, the interface compares the results of previous launches in an interesting way.

Branching strategy for large Qlik projects

Posted on by Alyona

Speed and flexibility of software development play an important role. However, with a large development team working at the same time, branching and merging code can get messy. These teams need to be provided with a process to implement changes concurrently. An effective branching strategy is a priority in resolving this issue.

A branching strategy is a set of rules that developers follow when writing, merging, and deploying code when using version control. In situations where multiple developers are working and adding changes at the same time, the branching strategy structures the repository and avoids merge confusion. Merge conflicts hinder rapid code delivery and hinder the creation and maintenance of an effective DevOps process.

Thus, the branching strategy creates a clear process for making changes to the system. This helps to solve this problem and provides the ability for multiple developers to work in parallel and independently to release faster and minimize conflicts.

The branching strategy allows to:

  • ensure coordination between developers and improve productivity;
  • parallel development;
  • implement planned and structured releases;
  • make changes to the software;
  • maintain bug-free code and the ability to quickly fix issues and push changes back into production without interrupting workflow.

The branching strategy is an important working tool. Arbitrarily creating branches in Git by different developers can lead to chaos. A branching strategy will allow to redirect attention to the development process itself, rather than version control.

Branches in Git are represented by tags or pointers. All changes are tracked as a directed acyclic graph, each node of which is a set of changes made simultaneously. Git branches provide developers with the ability to deviate from the master branch and create separate branches to isolate code changes. The default branch in Git is «main branch». The main advantage of a Git branch is its «light weight». That is, the data consists of snapshots series, and each time Git commits changes. It takes a snapshot of the current files state and keeps a link to this one. Such branches are not just copies, but also a pointer to the latest commit.

Basic Git branching strategies:

  • GitFlow is a model that shows the ability to develop in a team and individually. At the same time, only 2 main branches (master and develop) accompany the development life cycle. Other branches are temporary, which ensures a permanent order in the repository;
  • GitHub Flow is a simpler version. It only has master and feature branches. Consolidating all changes to master and deploying frequently allowed changes to be released immediately, following the principles of production and continuous delivery;
  • GitLab Flow – offers a dedicated production branch for managing code release to production. It is possible to configure automatic code deployment from this branch with each change in it. The release is provided through a merge from the master branch to production. With the help of this branch, it is possible to obtain data on the released code version. It is possible to determine the approximate release time by the time the merge commit was created;
  • Trunk-based development – an approach that involves development based on a single trunk branch. The key approach features are temporary branches existence (several days), continuous code review (the process of checking and analyzing the code by the developer before release), master readiness to release at any time. This approach ensures conflicts minimization during development, readiness for release without preparation, fast and high-quality feedback on PR/MR.

An example of a merge conflict and its resolution

Let's look at an example of a merge conflict and possible ways to resolve it using QOps to control versions of a Qlik Sense application. Let's take a Qlik Sense application and use QOps to create a source code repository and 2 branches dev1 and dev2 for individual developers. Imagine a situation where each of the developers changed some common element (for example, the background objects color with the output of KPI values).

Since the changes occurred on the same lines at the source level, when merging dev1 -> master and dev2 -> master, a merge conflict occurs in the last variant. Resolving a merge conflict requires deciding which of the options should be present in the reduced version. The VSCode development tool allows to interactively make this decision. However, this puts additional responsibility on the IT specialist who will perform the merging.

There is a risk of receiving from both developers code parts for a common element in a final application that may not be compatible. For example, after a merge, it is possible to get the following result when the first object is set to Accept Incoming and the second is set to Accept Current.

It is convenient to merge source codes using the web interface of version control systems. Below is the progress of a Pull Request on GitHub with a conflict-free merge.

The presence of conflicting changes leads to the output of the corresponding warning and additional information.

GitHub interface allows to resolve merge conflicts in small files, while the user needs to manually edit the source code marked as conflicting and confirm the changes. If this operation can’t be performed through the web interface, GitHub will ask to resolve conflicts locally.

Also, let's resolve the existing conflict in the opposite way. As a result, we get the expected result with the opposite coloring of common elements.