Eugeny Shtoltc - Notes of an IT Architect

* The technical architecture describes the implementation of the previous layers. It itself is also divided into layers, but these layers are not abstractions, but are a technical implementation in relation to the client. These layers are also called a stack, since they are not located from the user of the business layer one by one in a specific order and not a single layer can be skipped, while the layers of architecture describe the architecture in a different way, can be supplemented and changed, but we adhere to the set of layers accepted in TOGAF. So, the stack of WEB-applications consists of:

** Application layer – the layer with which users directly interact and which provide processes and is implemented using WEB-interfaces in browsers. Execution of business logic at the level of WEB-interfaces is unacceptable – all work is delegated to the underlying layer.

** The network layer ensures the operation of WEB-interfaces in the user's browsers, transmitting data. The upper layer of applications is needed by users who are comfortable using the graphical interface and so that any routine actions are performed for them. The network layer provides communication for server applications such as WEB servers and DBMS.

** The hardware layer is represented by runners. These devices can be hardware-based with varying degrees of versatility. For example, a load balancer can be purely hardware, it can be hardware with changeable firmware, it can be software executable on a general purpose computer of the x86 type, it can be launched both directly and in a virtual machine, and in a container – all these are implementation details.

** The storage layer is made up of storage devices. These devices can be specialized devices such as IBM DataPower or regular RAID with a control module. The data in it completely describes the state and is the result of work, and the previous layers are only needed to change and provide convenient access to users.

If necessary, other layers can be implemented, for example:

* information security layer implemented by the firewall;

* a layer of basic containers;

* layer of local fault tolerance (HA) using the example of the Kubernetes layer;

* containerization layer;

* virtualization layer;

* a layer of resilience to failures implemented by load balancers on different DataCenter.

In any case, the number of layers is standardized, those that differ are indicated, so that each layer belongs to a specific info-structural department and an operation department.

Let's pay more attention to the integration architecture, since this is the most critical layer for the architecture. In this layer, connections can be presented both in a graphical form (in the arrows on the diagrams between systems), and in a tabular form – in the form of a description of the supplier, consumer and contract (the supplier's obligations to the contractor). The arrows point from the supplier to the consumer, that is, in the direction of the integration flow, while the service modules are not indicated. Depending on whether the parameters are functional or non-functional, they will be described either by API or SLA. Also, depending on whether the connections are inside or from outside to inside and outside. The first type is more visual, and the second allows you to give more detailed characteristics.

The system itself can be integrated in different ways, such as:

* direct integration (communication via API point-to-point, advantages: minimum overhead, disadvantages: two-way revision of systems is required, complexity of change management, complexity of scaling, no reuse);

* using gateways (communication through the API of an integration layer, such as a queue with a firewall, advantages: minimum overhead, unified API, disadvantages: complexity of change management, complexity of scaling, no reuse);

* Enterprise data bus or enterprise service bus (ESB) provides asynchronous umbrella integration based on the principles of event and service approach (SOA, service-oriented architecture). The corporate data bus is able to flexibly route messages from one service to another. (advantages: unification, reusability due to SOA, replaceability of services due to SOA, disadvantages: an expensive solution in many applications, delivery time from tens of milliseconds);

* Service Mesh, like ESB, is umbrella, but applications do not need to integrate with it, since applications running in a containerized environment immediately receive integration. (microservices, advantage: minimum overhead, not noticeable for application developers);

* Integration file gateways and point-to-point file transfer (file overload). Point-to-point file transfer is the same point-to-point transfer, but it allows you to transfer large data in exchange for the transmission speed (advantage: it is possible to transfer very large amounts of data, high delivery guarantee, weak connectivity of integrated systems, greater control, broadcast mode, disadvantages: transmission speed, possibility of desynchronization, high security requirements). Communication protocols are CIFS (Common Internet File System), NFS (Network File System extends the local file system) and S3 (Simple Storage Service provides access to object storage such as Minio and Ceph) and transfer protocols HTTPS (HTTP + SSL), SFTP (SSH + SSL) and FTPS (FTP + SSL). From the point of view, records can be divided into block (disk) and object (writing to the key-value database: Bucket).

Systems can communicate in various ways:

* integration request (normal synchronous request-response),

* remote procedure call (RPC),

* sending a command to the queue (from the supplier to the consumer directly through the event queue),

* publish-subscribe, Push-Sub (sending events to a common queue, from which groups of events are retrieved from the system in advance, undefined by the provider),

* packet data transmission,

* transferring files to storage,

* streaming data.

The interactions themselves should be described, and preferably unified. To describe functional or non-functional parameters (response time, availability, message size, bandwidth) of interactions between the supplier and the consumer, a Contract is used, which the supplier undertakes to fulfill. Functional parameters are described using the Application Programming Interface (API). Service APIs can be divided into groups based on message format (DTO, JSON, XML, binary) or protocol (HTTP, REST, SOAP). It is important to specify in the API contract: ID, name, version, purpose, template, specification of input and output parameters. The API itself will contain methods (encouraging the consumer to take data, change, allocate, etc.). Parameters passed in a method are described by a method specification, for example, using OpenAPI or Swagger. For many languages, and primarily for the Java language, you can automatically generate a specification for OpenAPI using Swagger by Javadoc annotations (by special comments) in the code. The specification will be displayed in both text formats (JSON and YAML) and graphical. For ease of design, you can use the Swagger Editor (https://swagger.io/tools/swagger-editor/). This helps to organize automated contract testing. In general, the presence of the specification helps to organize API-first development, when the API specification is first written, and already the application code is developed for it, implementing it.

To bind an API to the system providing it, the endpoint concept is used. Endpoint is described either by an address with a port, or MMT, hiding them. APIs are described by a machine-readable description (declaration), for example Java interface and WSDL document.

The API life cycle is close in stages to the software life cycle and the following cycle stages can be outlined: requirements gathering, interface design, implementation, testing, operation, and decommissioning stages: decommissioning warning (marked as obsolete to avoid new users), decommissioning (tracking the decrease of existing users), disposal (closing access to the API and removing it), and the stage of creating a new version. The new version of the API can be compatible and incompatible with the old one, for this you can differentiate them by semantic versioning into incompatible with the previous version, compatible with the previous version and bug fixes. By semantic versioning, the version can be written as "major.minor.patch", for example 1.9.0 – > 1.10.0 – > 1.11.0.

The solution integration layer is described in the conceptual architecture. The conceptual framework only indicates integration. It is intended for coordination with stakeholders: architects of the integrated solution, corporate architect responsible for the landscape, product owners, owner of the variable landscape, security service, resource provision service (infrastructure department) for its quick approval. The content is indicated in the target architecture (detailed architecture) of the solution. She participates in the early stages of creating a service and in the acceptance tests of the solution within the framework of architectural control. The conceptual architecture is the source of artifacts for implementing and developing a detailed solution architecture. We can distinguish the following stages of creating a conceptual architecture: planning, creation, development of an incremental draft (sketch), approval, revision. In order for the diagrams to be understood by both the developer and the owner of the product, a short description (Architecture Decision Records, ADR) is needed: the goal, the essence of the solution, rejected options, rationale, consequences, affected systems, a link to the diagram and the originator's contacts.

The corporate data bus is usually used to connect individual systems. For connections within the system – point-to-point network connection and their control using integration gateways. Integration gateways can and can be used between systems, but often, these systems are not protocol compatible and require their transformation, which is provided by the corporate data bus, but not provided by the integration gateway. And the use of a corporate data bus to connect the components of one system is expensive, since the asynchronous protocol will be loaded by other systems due to the fact that the message queue that implements it will be filled with messages from other systems and subsequent messages will wait for their sending, which causes delays in data transmission… For the exchange of large amounts of data between systems and within systems – distributed information storage.

With the layers we figured out, now, imagine the architecture as a layer cake. We can cut off a piece from it and narrow it down in more detail, but the number of layers in all pieces of this pie will be the same. By analogy, we can divide the corporate architecture into chunks. These pieces can be separate systems that are directly used on the business layer, or it can be a group of these applications developed by some department or department. The pie itself is the corporate architecture, and the piece is the solution (service) architecture. Enterprise architecture appears when we need to add new pieces to the pie, while the pieces are the architecture of the solution, and the rules according to which are made, for example, the number of layers – corporate architecture standards, cakes and cream – providing technologies for corporate architecture.

Corporate architecture is the architecture of the entire IT landscape of a company. A corporate architect is guided by the principles of creating architecture, a kind of constitution. The principles are described in the third section of TOGAF 9.2 under clause 20. They govern which requirements, for example, the principle of customer focus or lean manufacturing will meet. It is important that all participants (stakeholders) agree to adhere to the same principles. The principles are categorized according to their applicability to the architecture layers: business, data, application, and technology.