high-availability

AMQP/RabbitMQ - Process messages sequentially

夙愿已清 提交于 2019-12-22 11:12:40
问题 I have one direct exchange. There is also one queue, bound to this exchange. I have two consumers for that queue. The consumers are manually ack'ing the messages once they've done the corresponding processing. The messages are logically ordered/sorted, and should be processed in that order. Is it possible to enforce that all messages are received and processed sequentially accross consumer A and consumer B? In other words, prevent A and B from processing messages at the same time. Note: the

Spark Config Files

China☆狼群 提交于 2019-12-22 10:19:45
问题 Can someone share with me the spark-env.sh and spark-default.conf file content which needs to be set to run Spark on YARN as client or cluster ? Also should i store spark_assembly jar to the HDFS and then create a environment variable in the ~/.bashrc file ? I am unable to start my spark-shell with --master yarn-client command. Please help !! Update **Daemon in NN1 :** 2945 JournalNode 3137 DFSZKFailoverController 6385 Jps 3338 NodeManager 22730 QuorumPeerMain 2747 DataNode 3228

What's best practice for HA gearman job servers

和自甴很熟 提交于 2019-12-20 19:38:14
问题 From gearman's main page, they mention running with multiple job servers so if a job server dies, the clients can pick up a new job server. Given the statement and diagram below, it seems that the job servers do not communicate with each other. Our question is what happens to those jobs that are queued in the job server that died? What is the best practice to have high-availability for these servers to make sure jobs aren't interrupted in a failure? You are able to run multiple job servers

Zero downtime deployment for Java apps

拟墨画扇 提交于 2019-12-20 10:39:09
问题 I am trying to build the very lightweight solution for zero downtime deployment for Java apps. For the sake of simplicity lets think that we have two servers. My solution is to use: On the "front" -- some load balancer (software) - I am thinking about HAProxy here. On the "back" - two servers, both running Tomcat with deployed application. When we are about to deploy new release We disable one of the servers with HAProxy, so only one server (let's call it server A, which is running old

ZooKeeper alternatives? (cluster coordination service) [closed]

情到浓时终转凉″ 提交于 2019-12-20 07:58:15
问题 Closed. This question is off-topic. It is not currently accepting answers. Want to improve this question? Update the question so it's on-topic for Stack Overflow. Closed 3 years ago . ZooKeeper is a highly available coordination service for data centers. It originated in the Hadoop project. One can implement locking, fail over, leader election, group membership and other coordination issues on top of it. Are there any alternatives to ZooKeeper? (free software of course) 回答1: I've looked

Spring Batch correctly restart uncompleted jobs in clustered environment

余生长醉 提交于 2019-12-19 21:24:42
问题 I used the following logic to restart the uncompleted jobs on single-node Spring Batch application: public void restartUncompletedJobs() { try { jobRegistry.register(new ReferenceJobFactory(documetPipelineJob)); List<String> jobs = jobExplorer.getJobNames(); for (String job : jobs) { Set<JobExecution> runningJobs = jobExplorer.findRunningJobExecutions(job); for (JobExecution runningJob : runningJobs) { runningJob.setStatus(BatchStatus.FAILED); runningJob.setEndTime(new Date()); jobRepository

How do I cluster ServiceMix?

孤者浪人 提交于 2019-12-18 13:17:11
问题 I am looking for some initial pointers on how to cluster a ServiceMix solution. Basically what I need is: having 2 (or more) ServiceMix instances serving my routing needs and sharing the load if one instance fails, other(s) continue to serve if the failed one is brought back to life, it joins the party Searching for information confuses me since some references (eg. http://trenaman.blogspot.fi/2010/04/four-things-you-need-to-know-about-new.html) talk about "JBI cluster engine". I don't want

Why are RDBMS considered Available (CA) for CAP Theorem

非 Y 不嫁゛ 提交于 2019-12-13 11:53:56
问题 If I understand the CAP Theorem correctly, availability means that the cluster continues to operate even if a node goes down. I've seen a lot of people (http://blog.nahurst.com/tag/guide) list RDBMS as CA, but I do not understand how RBDMS is available, as if a node goes down, the cluster must go down to maintain consistency. My only possible answer to this has been that most RDBMS are a single node, so there is no "non-failing" node. But, this seems to be a technicality, not true

TCP Proxy: Hold connections while backend is unavailable

一笑奈何 提交于 2019-12-12 21:29:46
问题 In the context of a Docker setup, I want to use an ambassador-like pattern to allow some container (e.g., database server) to restart gracefully without having to restart all dependent containers (e.g., web server) and without error messages because "database server is not available". Therefore, I was wondering: Is there a TCP proxy software (that would run in the ambassador container) that can accept and hold connections if the backend is down and re-establish them when it is back? In the

Achieve Fault Tolerance with Consul Cluster

坚强是说给别人听的谎言 提交于 2019-12-12 15:17:15
问题 I have created consul server cluster using different ports in localhost. I used below commands for that. server 1: consul agent -server -bootstrap-expect=3 -data-dir=consul-data -ui -bind=127.0.0.1 -dns-port=8601 -http-port=8501 -serf-lan-port=8303 -serf-wan-port=8304 -server-port=8305 -node=node1 server 2: consul agent -server -bootstrap-expect=3 -data-dir=consul-data2 -ui -bind=127.0.0.1 -dns-port=8602 -http-port=8502 -serf-lan-port=8306 -serf-wan-port=8307 -server-port=8308 -node=node2