问题
I have a question about the architecture of application composed of microservices.
I have few microservices, but those that are interesting in the context of this question are:
- Human Resources - here are stored all the user data such as username, sex, user's experience etc.
- Jobs - here are stored all the data about each job advertisement - job description, expected experience etc.
So, I need to create new microservice (let's name it Recommender) which sole purpose will be - based on the texts of the microservices mentioned above to recommend best fitted human account from Human Resources to each job ad. from Jobs. The NLP algorithm which I use to calculate the similarities need to walk each Human Resources item for every Jobs ad.
So far, so good. One way to achieve this is to introduce Message Queue. On each create/update operation of human resource from Human Resources to push new message with all the resource data through this MQ. The Recommender microservice will handle this new message, it will process the needed data (remove the not needed data) and it will store it in local DB. Same thing will be needed when Jobs resource is updated/created in order to calculate one-to-one similarities.
Here are my questions:
- Is there a better way to achieve the above?
- I know that best practices in the microservice architecture design told us that the data shouldn't be duplicated, but is it ok to store processed and limited data which is something like a copy of the original data?
Thanks!
回答1:
I think that the general architecture with the three services (HR, Jobs, Recommender) is fine as it clearly defines the responsibilities and separates the different functions in the system.
In my opinion, you should not keep any data persistently in the Recommender service. Imagine if the database schema for the HR database changes (e.g. new data field), you will perhaps have to change the recommender algorithm, but you want to change the database in only one (the HR) service, and not in both services. Imagine if for some reason the Recommender service missed an update event due to an error, you would somehow have to repair the inconsistent database state between the two services.
On the other hand, if you had a requirement to save on communication bandwidth between your services, you could think of keeping some data as an architectural tradeoff - but this seems improbable and would be kind of against the idea behind a microservices architecture.
回答2:
You asked two main points. I have experienced a similar situation, below are my recommendations. 1- Is there a better way to achieve the above? Answer- You are on the right path. Just ensure that you strictly follow the Microservice architecture and you deploy "Recommender" Microservice separately.
2- I know that best practices in the microservice architecture design told us that the data shouldn't be duplicated, but is it ok to store processed and limited data which is something like a copy of the original data? Answer- Microservice brings a lot of flexibility. You can very much have your own processed data in a dedicated data-store for the "Recommender" microservice, which will feed the main application with right matched candidate info. The issue you might face or want to plan for is what happens to the data consistency of your Recommender Microservice when anything changes in the source data stores "Human Resources" and "Jobs". I.e. you may want to re-run the Recommender logic to re-generate the results. You have used MQ to submit the job, but I would suggest using events that can be published from Source tables if key info changes and then Recommender being a Subscriber can listen to those events, fetch the data and re-run the logic to produce a new set of result(s).
来源：https://stackoverflow.com/questions/59143180/is-it-bad-idea-to-share-almost-identical-data-among-microservices-for-item-rec