Monthly Archives: April 2020

Print stacktraces for all threads on shutdown

debugging, JAVA, JVM, Scala

If your microservice stops responding from time to time, and they only way out is to kill it with SIGINT or SIGTERM then adding a shutdown hook might be the way to go. Do note that this will not work if you kill the process with SIGKILL (-9), cause that will result in an unclean shutdown.

Some of this code is heavily influenced by Print all of the thread’s information and stack traces : Exception « Development « Java Tutorial. But has been translated into Scala, and cleaned up a little.

The output would look something like this

 

Apache Zeppelin, with Spark and Cassandra, the perfect tool

Uncategorized

Zeppelin has become one of my favourite tools in my toolbox. I am heavily designing stuff for Cassandra and in Scala, and even though I love Cassandra there are times when things just gets so complicated with the CQL command line, and creating a small project in IntelliJ just seems like too much hazel. Then using Zeppelin to try out is just perfect. So this page is a How-To with some useful Cookbook recipes.

Setting Up Zeppelin

I use Docker where things are so much easier, and I pick v0.8.0 cause I never got 0.8.2 to work for some reason.

Download and Start Cassandra

 

Download and Start Zeppelin

Download Zeppelin image

Start Zeppelin on port 8080

-p hp:cp
hp = Host Port, the port on your local machine
cp = Container Port, the port inside the docker which is what Zeppelin is exposing

Go to localhost:8080 in your web browser and you should see something like this

Setup Zeppelin

Find out the IP address of Cassandra in you Docker network, as you can see of the inspect, the IP address is 172.17.0.3.

 

Set up IP address for Cassandra in the Spark Interpreter

Go to the section on “Spark”

Now add a row that says

Now also edit the Dependencies

You can do this in many ways, either you specify the MAVEN repo with version OR you download the JAR file(s) to disk and copy them into the Docker. I had to do the latter due to some issue with my network.

You need these two libraries :

Simply click on the JAR file and download the file, then copy it into the docker with

Setup IP address for Cassandra in Cassandra Interpreter

Create your first Notebook

Cookbook Recipes

Load Table into RDD and count rows

This is just to show how you load a table into an RDD, once in the RDD you can play around with it and do lots of stuff.

Show key spaces using the built in Cassandra interpreter using CQL

The result :

Create Keyspace and Table using CQL

Insert data by hand using CQL

Fill the table with bogus data using Spark and Scala

 

Select data using CQL

Create VIEW so that we can run SQL

 

Run SQL, ohh sweet SQL 🙂

By creating temporary views like this, we can also do joins if we would like to.

Obviously this is not how Cassandra was intended to be used, but the point here is more of giving the ability to troubleshoot, turist around in the data with ease instead of setting up a project, and do the joins inside of the code. Here we are able to really trail and error until we get what we want.

That was all for now

-Tobias