Common Integration Approaches

-

Common Integration approaches

You can see these approaches used.

File System 

One application will write the data needed by others to file system. Consuming systems will read the data from file and process accordingly.
This is probably oldest pattern used for sharing data between systems. This needs lot of maintenance. Both applications need to come to an agreement on when data will be written, when it will be consumed and who will manage the files. There is always a possibility of data corruption when it is in the file system. Human intervention is needed when there is a problem with the data. There is also a possibility of someone editing the file and change data. 

Common Database

Two applications share a common database. Or application will write to a common database or data warehouse. Data is read from the common database by other apps.
This pattern is difficult to maintain. This can cater to few systems. The moment you add more applications complexity increases. You are sharing more data than needed and you are slowly loosing control of data. Unless rights are configured correctly there is a possibility of data corruption.

Web service/ API 

Consuming application will call the web service or API exposed. This approach is comparatively better. Application can control what is being exposed. Data cannot be manipulated.
One issue with this approach is source system needs to take lot more load. Source system needs to add new API's for downstream systems.

Messages

One application writes the data needed by other applications into a Que system (or Enterprise service bus). Other applications consume these. Maintaining a que or service bus is expensive. Typically business need for having a message system are different than just data integration.
If the business requires an enterprise service bus then it is a good approach for integrating applications. You should use it only for critical needs where it is a must.

Comments

Post a Comment

Popular posts from this blog

Greedy computing with API

-
Image
 We all know computing resources have a cost. Memory, long term storage, transportation and computing all have their own costs. There are lot of scenarios where your API end point won't be able to respond in time. These are scenarios where you need to use more resources to provide a response. Let us be greedy with our resources and explore different patterns which can solve this. Polling  This is a commonly used pattern where the caller initiates a process. Then keeps on checking if the process is complete. If the process is complete then the caller reads the data needed.  This is a good pattern when both systems are part of the same organization. At least developers of caller and callee should be able to discuss this implementation.  Call back This is an improvement over polling pattern. Here the caller supplies a call back URL to the long running process. Once the process is complete it calls the callback URL and informs the caller that process results are availabl...
Read more

ChatGPT for cavemen

-
Image
I heard a lot about ChatGPT and thought to come out of my cave to find out what the hype is about. Is it going to eat all my lunch or eat me for lunch?  Like any product I gave it all my details and signed up. I asked it about quantum computing an it gave some answer. So far so good. Next day I wanted to check its blogging skills. So I opened chatGPT and I got this . I checked throughout the day and it is still down. So it won’t eat me till it is fed few more servers.  Continuing after it was up.  I asked it to write a blog about itself. Here is what was generated by chatGPT.
Read more

Event driven architecture for micro services

-
This is a work in progress. I am designing an event driven architecture to be used in systems using micro services. Micro service architecture is popular because it solves few problems in development process. While it solves some problems it also brings in new problems to the table. Bad designs impact micro services harder than monoliths. But this is thought is for another time.  How micro services talk to each other. In a monolith you have one gigantic database, or you can easily call another component to process the information. In a microservice you can use a Que (or event bus) to pass the information. This is the problem I want to discuss. Main issue with que is it is not designed for this scenario. There are few products which give a good solution by providing a subscription-based approach. But these are large implementations. Using these is similar to bringing an excavator to plant a sapling in your garden. Yes, many people go with using these large implementations to resolve...
Read more