Here are some notes on how to get on with Lua scripting for KrakenD

Imagine you have a backend that returns a JSON like this

So we route

The most basic KrakendD config would be something like this

In fact you could remove the ‘output_encoding’ and ‘encoding’, cause those are ‘json’ by default but gives clarity to the example.

Remove unwanted fields from the JSON

Let’s say that we do not want to expose the field “databaseId” Then we can apply a Lua script in KrakenD that solves this for us The configuration in krakendD can look something like this

The Lua code we place in a file on the same directory as the krakenD config

And the Lua function could look something like this

Dump some request info and the Data-object to the KrakenD log

Let’s place the lua script to dump some info on the ‘proxy’ level, the config could look something like this

The Lua code

Set or Modify the body when it is Content-Type : text

Note that KrakenD needs to be told how to interpret the data, and above the encoding was set explicitly to ‘json’ here we will set it to ‘string’ (i.e. text). The KrakenD config could look something like this

The Lua code below

Note that when we would like to modify the body and the data-object key will be ‘content’.

Set the http header Content-Type from Lua script

Imagine that the encoding/output_encoding was set to ‘string’ then the return data from krakenD will set the Content-Type to ‘text/plain; charset=utf-8’

If you would for some reason get a string and would like to set the Content-Type to ‘application/json; charset=utf-8’

You can do this from a Lua script like this

Bless you !
-Tobias

Local Port Forwarding

When you want to access your MySQL database from home, but only have an ssh connection.

I find my self reading up this time and time again, I guess I do not do it often enough, so I decided to write this down atleast for my own, but perhaps there is someone else also in the need of this small example/tutorial, so let’s put it out on the internet for all to see.

The example below intend to show how I can connect from my local laptop at home, to my office where an ssh connection has been allowed through a port forward in the Router to a machine (raspberry pi), and then I connect to a database on another machine in the same network. See picture below.

The ssh command looks like this

ssh -L 9900:192.168.0.30:3306 -p 21022 ssh-user@88.123.234.100

What the command does

ssh -L 9900:192.168.0.30:3306 -p 21022 ssh-user@88.123.234.100

1. Connect to the Router at 88.123.234.100 (which is a public IP)
2. The router listens on port 21022 , and will forward the connection to port 22 on 192.168.0.20 (which is the Raspberry Pi’s ssh server)
3. The ssh server on the Raspberry Pi will forward the request once more to 192.168.0.30 (the Linux PC) on port 3306, which is the port of MySQL.
4. Port 9900 is a local port on my laptop, and connecting my database client (e.g. dbeaver) to localhost:9900 will forward it to the MySQL process running port 3306 on 192.168.0.30

That is it !
/Tobias

Base config

Remove nested field

the json looks something like this

with this config the field “entitet / attribut / mottagare” can be removed

Convert JSON field to string

If we would like to convert a field that is a json object into a string, we can use the ruby module to make it a string instead

The output now looks like this

Sometimes you need to generate new timestamps or id’s or … for every new request you send with POSTMAN. The solution is to use pre-request scripts. These scripts are Javascripts that will use macros to populate values in your body content (json in my case).

With the following JSON

And the following pre-request script

then you can get the following result

JQ is a really good tool when you have lot of JSON and you just interested in some of that data.

Only see some attributes

Imagine you have a JSON like this, and you only want to see the “name” attribute

(extract specific attributes from a json array with objects, from array to array)

the JQ query would then be something like this

[.[] | { “email” : .email } ]

and that would give you the following output

You can use jqpplay, JQ Kung Fu and other online alternatives to try it out, and ofcourse the command line ‘jq’ command.

Just a list of strings for one “column” / attribute

If you want to just create an array of strings, then you try this jq query

[.[] | .email ]

Which would give you the following

JSON to CSV

You will the following output with lots of quotes

That is not so nice, so to avoid the quotes then use the “Raw Output” function
then the output will look like this

Much better 🙂

Trying to write down some of the things I did to get this working

What Shelly 3em publishes

You can see all the topics of the mqtt broker by doing like this :
mosquitto_sub -d -v -t “#”
The ‘#’ means ALL topics, and by doing this I will get something like this :

Install Node Red

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

Install Mosquito

sudo apt update && sudo apt upgrade
sudo apt install -y mosquitto mosquitto-clients
sudo systemctl enable mosquitto.service
mosquitto -v

show all what comes in regardless of topic (show all topics,, kind of…)
mosquitto_sub -d -v -t “#”

cat /etc/mosquitto/mosquitto.conf
sudo vi /etc/mosquitto/mosquitto.conf

sudo systemctl restart mosquitto

Install PostgreSQL

(*) Do not forget to setup remote access to postgresql

Create database, schema and table

Node Red

Insert data into PostgreSQL

Download Docker image

docker pull python

Start up Docker image with just a bash shell as entrypoint

docker run -it –rm –name python python /bin/bash

now you should see something like this :

root@ba17e57ff5e8:/# 
root@ba17e57ff5e8:/# 
root@ba17e57ff5e8:/# python -V
Python 3.10.4

Attach to a running docker container

docker exec -it <container-name> bash

Exposing PORT to the host outside the docker

docker run -p 90:80 -it –rm –name python python /bin/bash

Port 80 is the port used INSIDE the Docker
Port 90 is the port on the host (so OUTSIDE)

This means that on the laptop I would do

curl http://localhost:90

and reach the webserver running inside the docker listening on port 80.

So how do you serialize NULL ?
NULL would typically mean that the attribute is omitted from the json, but what if you WANT the NULL to be there, to symbolize an attribute that should be REMOVED.

    @AllArgsConstructor
    @NoArgsConstructor
    @Getter
    @Setter
    @ToString
    public static class Product {
        public String model;
        public String color;
        public String shirtSize;
    }

    @Test
    void howToSerializeNULL() {
        Product bossShirt = new Product("super slim", "red", "xl");
        System.out.println("Product : " + JSONUtils.stringify(bossShirt));
        bossShirt.setShirtSize(null);
        System.out.println("Product : " + JSONUtils.stringify(bossShirt));
        Map<String, Object> obj = new HashMap<>();
        obj.put("model", "super slim");
        obj.put("color", "red");
        obj.put("shirtSize", JSONObject.NULL);
        System.out.println("Product : " + JSONUtils.stringify(obj));
        GsonBuilder builder = new GsonBuilder();
        builder.serializeNulls();
        Gson gson = builder.create();
        System.out.println("Product : " + gson.toJson(obj));
        System.out.println("Product : " + gson.toJson(bossShirt));
    }

The purpose of this post is simply to give a hint on how to use Mockito, Spy, and JUnit 5.

package se.tkartor.microservice.tols;

import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.mockito.Mockito;
import org.mockito.junit.jupiter.MockitoExtension;

@ExtendWith(MockitoExtension.class)
public class MockTest {

    @Mock
    Car mockCarTWO;

    @Test
    public void noMockJustAssertTest() {
        Car car = new Car("red", 250, new Wheels(19));
        Assertions.assertEquals(250, car.maxSpeed);
    }

    /**
     * The purpose of this test is to show that a Mock can be created
     * using the API and does not have to be created using the @Mock annotation
     * NOTE! that creating a mock out of a class, means that it still serves
     * the interface/API of the original class, but NONE of the methods
     * will do anything nor will the return anything
     * The mock-instance is simply an empty shell, and hence the last
     * commented-out code/line is not possible, since it will return NULL
     */
    @Test
    public void mockitoAPItest() {
        Car mockCar = Mockito.mock(Car.class);
        mockCar.setColor("blue");
        mockCar.setMaxSpeed(100);
        Mockito.verify(mockCar).setColor("blue");
        // Assertions.assertEquals("blue" , mockCar.getColor());
        Assertions.assertNull( mockCar.getColor() );
    }

    /**
     * Based on the fact that a mocked class is an empty shell
     * it is possible to also attach a.k.a spy on a instance
     * and carry out mocking this way, this is useful when you have
     * and instance that does everything right, except you need to
     * see what happens when you demand it to return a certain value
     * under certain cirumstances. The example below hopefully
     * explains this :-)
     */
    @Test
    public void spyTest() {
        Car mockCar = Mockito.spy(new Car("red", 90, new Wheels(19)));
        mockCar.setColor("blue");
        mockCar.setMaxSpeed(100);
        Mockito.verify(mockCar).setColor("blue");
        Assertions.assertEquals("blue", mockCar.getColor());
    }

    /**
     * Nothing new, as explained above
     * the instance of the mocked class will be an empty shell
     */
    @Test
    public void mockitoAPITest() {
        Car mockCar = Mockito.mock(Car.class);
        Mockito.when(mockCar.getColor()).thenReturn("green");
        mockCar.setColor("blue");
        mockCar.setMaxSpeed(100);
        mockCar.setWheels(new Wheels(19)); // this is possible since the api signature is there and hence the mock
        // allows it to be called, but it does not do anything
        Mockito.verify(mockCar).setColor("blue");
        Assertions.assertEquals("green", mockCar.getColor());
        Assertions.assertNull(mockCar.getWheels()); // This is null, since there is no mock for it
    }

    /**
     * The purpose with this test was simply
     * to use the @Mock annotation insead of the
     * mockito API, a somewhat lightweight / easy to read approach
     */
    @Test
    public void mockitoAnnotationTest() {
        Mockito.when(mockCarTWO.getColor()).thenReturn("green");
        Assertions.assertEquals("green", mockCarTWO.getColor());
    }

    public static class Wheels {
        private int size;

        public Wheels(int size) {
            this.size = size;
        }

        public int getSize() {
            return size;
        }

        public void setSize(int size) {
            this.size = size;
        }
    }

    public static class Car {
        private String color;
        private long maxSpeed;
        private Wheels wheels;

        public Car(String color, long maxSpeed, Wheels wheels) {
            this.color = color;
            this.maxSpeed = maxSpeed;
            this.wheels = wheels;
        }

        public int wheelSize() {
            return wheels.getSize();
        }

        public String getColor() {
            return color;
        }

        public void setColor(String color) {
            this.color = color;
        }

        public long getMaxSpeed() {
            return maxSpeed;
        }

        public void setMaxSpeed(long maxSpeed) {
            this.maxSpeed = maxSpeed;
        }

        public Wheels getWheels() {
            return wheels;
        }

        public void setWheels(Wheels wheels) {
            this.wheels = wheels;
        }
    }
}