mesosphere

I have a private Docker registry that is accessible at https://docker.somedomain.com (over standard port 443 not 5000). My infrastructure includes a set up of Mesosphere, which have docker containerizer enabled. I'm am trying to deploy a specific container to a Mesos slave via Marathon; however, this always fails with Mesos failing the task almost immediately with no data in stderr and stdout of that sandbox. I tried deploying from an image from the standard Docker Registry and it appears to work fine. I'm having trouble figuring out what is wrong. My private Docker registry does not require

I installed Mesos in an OpenStack environment using these instructions from Mesosphere: https://open.mesosphere.com/getting-started/datacenter/install/ . I ran the verification test as described and it was successful. UI for both Mesos and Marathon are working as expected. When I run the Spark shell from my laptop I cannot connect. The shell hangs with the output below. I don't see anything in the Mesos master or slave logs that would indicate an error, so am not sure what to investigate next. Any help would be appreciated. TOMWATER-M-60SN:bin tomwater$ ./spark-shell --master mesos://zk://10

I'm running DC/OS 1.7 with marathon-lb. spray.io 1.3.3 is returning 400 to all marathon-lb/HAProxy heath check calls: request has a relative URI and is missing a Host header so marathon-lb never routes any requests to the service. The health check in the marathon json is: "healthChecks": [ { "path": "/health", "protocol": "HTTP", "portIndex": 0, "gracePeriodSeconds": 10, "intervalSeconds": 2, "timeoutSeconds": 10, "maxConsecutiveFailures": 10, "ignoreHttp1xx": false } ], and the logging by spray.io in the docker container is: [WARN] [08/19/2016 23:53:42.534] [asp-service-akka.actor.default

In slide 25 of this talk by Twitter's Head of Open Source office, the presenter says that Mesos allows one to track and manage even GPU (I assume he meant GPGPU) resources. But I cant find any information on this anywhere else. Can someone please help? Besides Mesos, are there other cluster managers that support GPGPU? Mesos does not yet provide direct support for (GP)GPUs, but does support custom resource types. If you specify --resources="gpu(*):8" when starting the mesos-slave, then this will become part of the resource offer to frameworks, which can launch tasks that claim to use these

We have set up a test cluster with Mesosphere on AWS, in a private VPC. We have some Docker images which are public, which are easy enough to deploy. However most of our services are private images, hosted on the Docker Hub private plan, and require authentication to access. Mesosphere is capable of private registry authentication, but it achieves this in a not-exactly-ideal way: a HTTPS URI to a .dockercfg file needs to be specified in all Mesos/Marathon task definitions. As the title suggests, the question is basically: how should the .dockercfg file be hosted within AWS so that access may

HDFS is not necessary but recommendations appear in some places. To help evaluate the effort spent in getting HDFS running: What are the benefits of using HDFS for Spark workloads? Spark is a distributed processing engine and HDFS is a distributed storage system. If HDFS is not an option, then Spark has to use some other alternative in form of Apache Cassandra Or Amazon S3. Have a look at this comparision S3 – Non urgent batch jobs. S3 fits very specific use cases, when data locality isn’t critical. Cassandra – Perfect for streaming data analysis and an overkill for batch jobs. HDFS – Great

Can Apache Mesos 'master' nodes be co-located on the same machine as Mesos 'slave' nodes? Similarly (for high-availability (HA) deploys), can the Apache Zookeeper nodes used in Mesos 'master' election be deployed on the same machines as Mesos 'slave' nodes? Mesos recommends 3 'masters' be used for HA deploys, and Zookeeper recommends 5 nodes be used for its quorum election system. It would be nice to have these services running along side Mesos 'slave' processes instead of committing 8 machines to effectively 'non-productive' tasks. If such a setup is feasible, what are the pros/cons of such a