Cassandra host in cluster with null ID

Question

Note: We are seeing this issue in our Cassandra 2.1.12.1047 (DSE 4.8.4) cluster with 6 nodes across 3 regions (2 in each region).

Trying to update schemas on our c

Answer 1

I've never had to do this myself, but probably the only thing left for you to do is to assassinate the endpoint. This was made into a nodetool command (nodetool assassinate) in Cassandra 2.2. But prior to that version, the only way to do it is via JMX. Here's a Gist with detailed instructions (instructions and code by Justen Walker).

Prerequisites

1. Log onto existing cluster alive node

2. Download JMX Term

wget

$ wget -q -O jmxterm.jar
> http://downloads.sourceforge.net/cyclops-group/jmxterm-1.0-alpha-4-uber.jar
> curl

or

 $ curl -s -o jmxterm.jar
 http://downloads.sourceforge.net/cyclops-group/jmxterm-1.0-alpha-4-uber.jar

3. Run jmxterm

$ java -jar ./jmxterm.jar
Welcome to JMX terminal. Type "help" for available commands.
$>

Assassinate node

Example bad node: 10.0.0.100

• Connect to the local cluster
• Select the Gossiper MBean Run the unsafeAssassinateEndpoint with the ip of the bad node

$>open
localhost:7199
#Connection to localhost:7199 is opened 

$>bean org.apache.cassandra.net:type=Gossiper
#bean is set to org.apache.cassandra.net:type=Gossiper

$>run unsafeAssassinateEndpoint 10.0.0.100
#calling operation unsafeAssassinateEndpoint of mbean org.apache.cassandra.net:type=Gossiper
#operation returns: null 

$>quit

Update 20160308:

I've never had to do this myself

Just had to do this myself. Totally looked-up and followed the steps in my own answer, too.