Apache Kafka vs RabbitMQ

RabbitMQ is an open-source message-broker software that originally implemented the Advanced Message Queuing Protocol (AMQP) and has since been extended with a plug-in architecture to support Streaming Text Oriented Messaging Protocol (STOMP), MQ Telemetry Transport (MQTT), and other protocols.

Written in Erlang, the RabbitMQ server is built on the Open Telecom Platform framework for clustering and failover. Client libraries to interface with the broker are available for all major programming languages. The source code is released under the Mozilla Public License.


In RabbitMQ, messages are stored until a receiving application connects and receives a message off the queue. The client can either ack (acknowledge) the message when it receives it or when the client has completely processed the message. In either situation, once the message is acked, it’s removed from the queue.

unlike most messaging systems, the message queue in Kafka is persistent. The data sent is stored until a specified retention period has passed, either a period of time or a size limit. The message stays in the queue until the retention period/size limit is exceeded, meaning the message is not removed once it’s consumed. Instead, it can be replayed or consumed multiple times, which is a setting that can be adjusted.


RabbitMQ supports several standardized protocols such as AMQP, MQTT, STOMP, etc, where it natively implements AMQP 0.9.1. The use of a standardized message protocol allows you to replace your RabbitMQ broker with any AMQP based broker.

Kafka uses a custom protocol, on top of TCP/IP for communication between applications and the cluster. Kafka can’t simply be removed and replaced, since its the only software implementing this protocol.

The ability of RabbitMQ to support different protocols means that it can be used in many different scenarios. The newest version of AMQP differs drastically from the officially supported release, 0.9.1. It is unlikely that RabbitMQ will deviate from AMQP 0.9.1. Version 1.0 of the protocol released on October 30, 2011 but has not gained widespread support from developers. AMQP 1.0 is available via plugin.

Pull vs Push approach

RabbitMQ is push-based, while Kafka is pull-based. With push-based systems, messages are immediately pushed to any subscribed consumer. In pull-based systems, the brokers waits for the consumer to ask for data. If a consumer is late, it can catch up later.


RabbitMQ’s benefits is the ability to flexibly route messages. Direct or regular expression-based routing allows messages to reach specific queues without additional code. RabbitMQ has four different routing options: direct, topic, fanout, and header exchanges. Direct exchanges route messages to all queues with an exact match for something called a routing key. The fanout exchange can broadcast a message to every queue that is bound to the exchange. The topics method is similar to direct as it uses a routing key but allows for wildcard matching along with exact matching.

Kafka does not support routing; Kafka topics are divided into partitions which contain messages in an unchangeable sequence. You can make use of consumer groups and persistent topics as a substitute for the routing in RabbitMQ, where you send all messages to one topic, but let your consumer groups subscribe from different offsets.

Message Priority

RabbitMQ supports priority queues, a queue can be set to have a range of priorities. The priority of each message can be set when it is published. Depending on the priority of the message it is placed in the appropriate priority queue. Here follows a simple example: We are running database backups every day, for our hosted database service. Thousands of backup events are added to RabbitMQ without order. A customer can also trigger a backup on demand, and if that happens, a new backup event is added to the queue, but with a higher priority.

A message cannot be sent with a priority level, nor be delivered in priority order, in Kafka. All messages in Kafka are stored and delivered in the order in which they are received regardless of how busy the consumer side is.


RabbitMQ was originally created by Rabbit Technologies Ltd. The project became part of Pivotal Software in May 2013. The source code for RabbitMQ is released under the Mozilla Public License. The license has never changed (as of Nov. 2019).

Kafka was originally created at LinkedIn. It was given open-source status and passed to the Apache Foundation in 2011. Apache Kafka is covered by the Apache 2.0 license. 


RabbitMQ has been in the market for a longer time than Kafka – 2007 & 2011 respectively. Both RabbitMQ and Kafka are “mature”, they both are considered to be reliable and scalable messaging systems.

Ideal use case

Kafka is ideal for big data use cases that require the best throughput, while RabbitMQ is ideal for low latency message delivery, guarantees on a per-message basis, and complex routing.


ToolApache KafkaRabbitMQ
Message orderingMessages are sent to topics by message key.
Provides message ordering due to its partitioning.
Not supported.
Message lifetimeKafka persists messages and is a log, this is managed by specifying a retention policyRabbitMQ is a queue, so messages are removed once they are consumed, and acknowledgment is provided.
Delivery GuaranteesRetains order only inside a partition. In a partition, Kafka guarantees that the whole batch of messages either fails or passes.Atomicity is not guaranteed
Message prioritiesNot supportedIn RabbitMQ, priorities can be specified for consuming messages on basis of high and low priorities


Apache Kafka vs IBM MQ

Message Queue (MQ)

A Message Queue (MQ) is an asynchronous service-to-service communication protocol used in microservices architectures. In MQ, messages are queued until they are processed and deleted. Each message is processed only once. In addition, MQs can be used to decouple heavyweight processing, buffering, and batch work.

Apache Kafka

Apache Kafka was originally developed at Linkedin as a stream processing platform before being open-sourced and gaining large external demand. Later, the Kafka project was handled by the Apache Software Foundation. Today, Apache Kafka is widely known as an open-source message broker and a distributed data storage platform written in Scala.

It provides services in a distributed, highly scalable, elastic, fault-tolerant, and secure manner. Options are available to self manage your kafka environments or fully managed services offered by vendors. It can be deployed on bare-metal hardware, virtual machines, and containers in on-premise as well as cloud environments.


IBM MQ is a messaging middleware that integrates various business applications and data across multiple platforms faster and easier. It provides enterprise-grade messaging capabilities with a proven record for expertly and securely moving data. Indeed, apps can communicate with the aid of IBM MQ. By transmitting message data via messaging queues, IBM MQ makes exchanging information easier for applications, services, systems, and files. This dramatically simplifies the process of developing and maintaining business applications.

Additionally, IBM MQ fits into several environments, such as on-premise, cloud, and hybrid cloud deployments, and is compatible with a broad range of computing systems. It also offers a global messaging backbone with a service-oriented architecture (SOA).


Initial set up for both IBM MQ & Kafka is straightforward and has good documentation & community support


Pull based communication is used in Kafka where receiving system send a message request to producing system. IBM MQ utilizes push based communication where it pushes the message to the queue where any receiver can consume at the same time from multiple systems


Kafka is an open-source solution. IBM MQ is a paid platform. IBM MQs has good customer support. Kafka on the other hand provides paid assistance based on subscription system but there is good open-source community as it is fairly popular messaging solutions


IBM MQ offers a range of advanced capabilities such as enhanced granular security and message simplification capability while Apache Kafka do not. However, both provide superior security features to build data sensitive, mission critical applications

In Apache Kafka, messages are not erased once the receiving system has read them. Hence, it is easier to log events


  • Both Kafka and MQ can be horizontally scaled. But Kafka is more scalable with the number of consumers as it uses the single log file for all consumers
  • IBM MQ is suited for applications which require high reliability and do not tolerate message loss where as Kafka is suited for applications which requires high throughput
  • Apache Kafka can get a message from one system to it’s receiver quickly compared to traditional message queue tools, but each receiver must make a request for the message, rather than the producing system placing the message into an accessible queue.  Additionally, while Apache Kafka can be used to log events and scales well, it doesn’t include as many granular features for security and message simplification. 


Apache Kafka – Java Producer & Consumer

Apache Kakfa is an opensource distributed event streaming platform which works based on publish/subscribe messaging system. That means, there would be a producer who publishes messages to Kafka and a consumer who reads messages from Kafka. In between, Kafka acts like a filesystem or database commit log.

In this article we will discuss about writing a Kafka producer and consumer using Java with customized serialization and deserializations.

Kakfa Producer Application:

Producer is the one who publish the messages to Kafka topic. Topic is a partitioner in Kafka environment, it is very similar to a folder in a file system. In the below example program, messages are getting published to Kafka topic ‘kafka-message-count-topic‘.

package com.malliktalksjava.kafka.producer;

import java.util.Properties;
import java.util.concurrent.ExecutionException;

import com.malliktalksjava.kafka.constants.KafkaConstants;
import com.malliktalksjava.kafka.util.CustomPartitioner;
import org.apache.kafka.clients.producer.*;
import org.apache.kafka.common.serialization.LongSerializer;
import org.apache.kafka.common.serialization.StringSerializer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class KafkaSampleProducer {

    static Logger log = LoggerFactory.getLogger(KafkaSampleProducer.class);

    public static void main(String[] args) {

    static void runProducer() {
        Producer<Long, String> producer = createProducer();

        for (int index = 0; index < KafkaConstants.MESSAGE_COUNT; index++) {
            ProducerRecord<Long, String> record = new ProducerRecord<Long, String>(KafkaConstants.TOPIC_NAME,
                    "This is record " + index);
            try {
                RecordMetadata metadata = producer.send(record).get();
                //log.info("Record sent with key " + index + " to partition " + metadata.partition() +
                 //       " with offset " + metadata.offset());
                System.out.println("Record sent with key " + index + " to partition " + metadata.partition() +
                        " with offset " + metadata.offset());
            } catch (ExecutionException e) {
                log.error("Error in sending record", e);
            } catch (InterruptedException e) {
                log.error("Error in sending record", e);

    public static Producer<Long, String> createProducer() {
        Properties props = new Properties();
        props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, KafkaConstants.KAFKA_BROKERS);
        props.put(ProducerConfig.CLIENT_ID_CONFIG, KafkaConstants.CLIENT_ID);
        props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, LongSerializer.class.getName());
        props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
        props.put(ProducerConfig.PARTITIONER_CLASS_CONFIG, CustomPartitioner.class.getName());
        return new KafkaProducer<>(props);

Kakfa Consumer Program:

Consumer is the one who subscribe to Kafka topic to read the messages. There are different ways to read the messages from Kafka, below example polls the topic for every thousend milli seconds to fetch the messages from Kafka.

package com.malliktalksjava.kafka.consumer;

import java.util.Collections;
import java.util.Properties;

import com.malliktalksjava.kafka.constants.KafkaConstants;
import com.malliktalksjava.kafka.producer.KafkaSampleProducer;
import org.apache.kafka.clients.consumer.Consumer;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.LongDeserializer;
import org.apache.kafka.common.serialization.StringDeserializer;

import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class KafkaSampleConsumer {
    static Logger log = LoggerFactory.getLogger(KafkaSampleProducer.class);

    public static void main(String[] args) {

    static void runConsumer() {
        Consumer<Long, String> consumer = createConsumer();
        int noMessageFound = 0;
        while (true) {
            ConsumerRecords<Long, String> consumerRecords = consumer.poll(1000);
            // 1000 is the time in milliseconds consumer will wait if no record is found at broker.
            if (consumerRecords.count() == 0) {
                if (noMessageFound > KafkaConstants.MAX_NO_MESSAGE_FOUND_COUNT)
                    // If no message found count is reached to threshold exit loop.

            //print each record.
            consumerRecords.forEach(record -> {
                System.out.println("Record Key " + record.key());
                System.out.println("Record value " + record.value());
                System.out.println("Record partition " + record.partition());
                System.out.println("Record offset " + record.offset());

            // commits the offset of record to broker.

        public static Consumer<Long, String> createConsumer() {
            Properties props = new Properties();
            props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, KafkaConstants.KAFKA_BROKERS);
            props.put(ConsumerConfig.GROUP_ID_CONFIG, KafkaConstants.GROUP_ID_CONFIG);
            props.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, LongDeserializer.class.getName());
            props.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName());
            props.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, KafkaConstants.MAX_POLL_RECORDS);
            props.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, "false");
            props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, KafkaConstants.OFFSET_RESET_EARLIER);

            Consumer<Long, String> consumer = new KafkaConsumer<>(props);
            return consumer;

Messages will be published to a Kafka partition called Topic. A Kafka topic is sub-divided into units called partitions for fault tolerance and scalability.

Every Record in Kafka has key value pairs, while publishing messages key is optional. If you don’t pass the key, Kafka will assign its own key for each message. In Above example, ProducerRecord<Integer, String> is the message that published to Kafka has Integer type as key and String as value.

Message Model Class: Below model class is used to publish the object. Refer to below descriptions on how this class being used in the application.

package com.malliktalksjava.kafka.model;

import java.io.Serializable;

public class Message implements Serializable{

    private static final long serialVersionUID = 1L;

    private String id;
    private String name;

    public String getId() {
        return id;

    public void setId(String id) {
        this.id = id;

    public String getName() {
        return name;

    public void setName(String name) {
        this.name = name;

Constants class: All the constants related to this application have been placed into below class.

package com.malliktalksjava.kafka.constants;

public class KafkaConstants {

    public static String KAFKA_BROKERS = "localhost:9092";

    public static Integer MESSAGE_COUNT=100;

    public static String CLIENT_ID="client1";

    public static String TOPIC_NAME="kafka-message-count-topic";

    public static String GROUP_ID_CONFIG="consumerGroup1";

    public static String GROUP_ID_CONFIG_2 ="consumerGroup2";

    public static Integer MAX_NO_MESSAGE_FOUND_COUNT=100;

    public static String OFFSET_RESET_LATEST="latest";

    public static String OFFSET_RESET_EARLIER="earliest";

    public static Integer MAX_POLL_RECORDS=1;

Custom Serializer: Serializer is the class which converts java objects to write into disk. Below custom serializer is converting the Message object to JSON String. Serialized message will be placed into Kafka Topic, this message can’t be read until it is deserialized by the consumer.

package com.malliktalksjava.kafka.util;

import java.util.Map;
import com.malliktalksjava.kafka.model.Message;
import org.apache.kafka.common.serialization.Serializer;
import com.fasterxml.jackson.databind.ObjectMapper;

public class CustomSerializer implements Serializer<Message> {

    public void configure(Map<String, ?> configs, boolean isKey) {


    public byte[] serialize(String topic, Message data) {
        byte[] retVal = null;
        ObjectMapper objectMapper = new ObjectMapper();
        try {
            retVal = objectMapper.writeValueAsString(data).getBytes();
        } catch (Exception exception) {
            System.out.println("Error in serializing object"+ data);
        return retVal;
    public void close() {


Custom Deserializer: Below custom deserializer, converts the serealized object coming from Kafka into Java object.

package com.malliktalksjava.kafka.util;

import java.util.Map;

import com.malliktalksjava.kafka.model.Message;
import org.apache.kafka.common.serialization.Deserializer;

import com.fasterxml.jackson.databind.ObjectMapper;

public class CustomObjectDeserializer implements Deserializer<Message> {

    public void configure(Map<String, ?> configs, boolean isKey) {

    public Message deserialize(String topic, byte[] data) {
        ObjectMapper mapper = new ObjectMapper();
        Message object = null;
        try {
            object = mapper.readValue(data, Message.class);
        } catch (Exception exception) {
            System.out.println("Error in deserializing bytes "+ exception);
        return object;

    public void close() {

Custom Partitioner: If you would like to do any custom settings for Kafka, you can do that using the java code. Below is the sample custom partitioner created as part of this applicaiton.

package com.malliktalksjava.kafka.util;

import org.apache.kafka.clients.producer.Partitioner;
import org.apache.kafka.common.Cluster;
import java.util.Map;

public class CustomPartitioner implements Partitioner{

    private static final int PARTITION_COUNT=50;

    public void configure(Map<String, ?> configs) {


    public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
        Integer keyInt=Integer.parseInt(key.toString());
        return keyInt % PARTITION_COUNT;

    public void close() {

Here is the GIT Hub link for the program: https://github.com/mallikarjungunda/kafka-producer-consumer

Hope you liked the details, please share your feedback in comments.

ReactJs basic application creation

React is one of the popular java script library used to build user interfaces in web applications. 

As per Reactjs.orgReact is a declarative, efficient, and flexible JavaScript library for building user interfaces. It lets you compose complex UIs from small and isolated pieces of code called “components”.

In this tutorial, we would like to understand on how to create a basic react application using command line. You need to have nodejs installed into your desktop or laptop to create reactjs based application.

Navigate to your workspace folder(folder where you want to create project) in command/terminal prompt. Type below command to create basic react application with the folder name as malliktalksjava.

npx create-react-app malliktalksjava

Move to react application folder using below command.

cd malliktalksjava/

Start the application using below command.

npm start

Access localhost URL, http://localhost:3000/. React basic application will be displayed as given below.

React JS_Hello World App

This basic application doesn’t handle back-end logic or databases; it just creates a front-end build pipeline, so you can use it with any back-end you want. But, it installs Babel and webpack behind the scenes, below is the purpose of Babel and webpack:

Babel is a JavaScript compiler and transpiler, used to convert one source code to other. You will be able to use the new ES6 features in your code where, babel converts it into plain old ES5 which can be run on all browsers.

Webpack is a module bundler, it takes dependent modules and compiles them to a single bundle file. You can use this bundle while developing apps using command line or, by configuring it using webpack.config file.


Apache Kafka – Environment Setup

Apache Kakfa is an opensource distributed event streaming platform which works based on publish/subscribe messaging system. That means, there would be a producer who publishes messages to Kafka and a consumer who reads messages from Kafka. In between, Kafka acts like a filesystem or database commit log.

In this post we will setup kafka local environment, create topic, publish and consume messages using console clients.

Step 1: Download latest version of Apache Kafka from Apache Kafka website: https://kafka.apache.org/downloads.

Extract the folder into your local and navigate to the folder in Terminal session (if Mac) or command line (if windows):

$ tar -xzf kafka_2.13-3.1.0.tgz 
$ cd kafka_2.13-3.1.0

Step 2: Run Kafka in your local:

Run zookeeper using the below command terminal/command line window 1:

# Start the ZooKeeper service
$ bin/zookeeper-server-start.sh config/zookeeper.properties

Run Kafka using the below command in another terminal or command line:

# Start the Kafka broker service
$ bin/kafka-server-start.sh config/server.properties

Note: You must have Java8 or above in your machine to run Kafka.

Once above two services are run successfully in local, you are set with running Kafka in your local machine.

Step 3: Create topic in Kafka to produce/consume the message in another terminal or command like. In below example, topic name is ‘order-details’ and kafka broker is running in my localhost 9092 port.

$ bin/kafka-topics.sh --create --topic order-details --bootstrap-server localhost:9092

If needed, use describe topic to understand more details about topic created above:

$ bin/kafka-topics.sh --describe --topic order-details --bootstrap-server localhost:9092

Output looks like below:
Topic: order-details	PartitionCount: 1	ReplicationFactor: 1	Configs: segment.bytes=1073741824
	Topic: order-details	Partition: 0	Leader: 0	Replicas: 0	Isr: 0

Step 4: Write events to topic

Run the console producer client to write a few events into your topic. By default, each line you enter will result in a separate event being written to the topic.

$ bin/kafka-console-producer.sh --topic order-details --bootstrap-server localhost:9092
Order 1 details
Order 2 details

Step 5: Read events from Kafka

Open another terminal session/command line and run the console consumer client to read the events you just created:

$ bin/kafka-console-consumer.sh --topic order-details --from-beginning --bootstrap-server localhost:9092
Order 1 details
Order 2 details


By completing all the above steps, you learned about setting up kafka environment, creating topics, producing the messages using console producer and consming the message using console consumer.

Spring Cloud Sleuth & Zipkin – Distributed Logging and Tracing

In standard applications, app logs are implemented into a single file which can be read for debugging purposes. However, apps which follows microservices architecture style comprises multiple small apps and multiple log files are to maintained to have at least one file per microservice. Due to this , identification and correlation of logs to a specific request chain becomes difficult.

For this, distributed logging & tracing mechanism can be implemented using tools like Sleuth, Zipkin, ELK etc

How to use Sleuth?

Sleuth is part of spring cloud libraries. It can be used to generate the traceid, spanid and add this information to the service calls in headers and mapped diagnostic context (MDC). These ids can be used by the tools such as Zipkin, ELK to store, index and process the log file.

To use sleuth in the app, following dependencies needs to be added


How to use Zipkin?

Zipkin contains two components

  • Zipkin Client
  • Zipkin Server

Zipkin client contains Sampler which collects data from ms apps with the help of sleuth and provides it the zipkin server.

To use zipkin client following dependency needs to be added in the application


To use zipkin server, we need to download and set up the server in our system

zipkin server

Implementation on microservice apps

To see distributed logging implementation, we need to create three services with the same configuration, the only difference has to be the service invocation details where the endpoint changes.

  • Create services as Spring boot applications with WebRest RepositoryZipkin and Sleuth dependencies.
  • Package services inside a single parent project so that three services can be built together. Also, I’ve added useful windows scripts in github repo to start/stop all the services with a single command
  • Below is one rest controller in service1 which exposes one endpoint and also invokes one downstream service using the RestTemplate. Also, we are using Sampler.ALWAYS_SAMPLE that traces each action.

Service 1

package com.mvtechbytes.service1;
import brave.sampler.Sampler;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.core.ParameterizedTypeReference;
import org.springframework.http.HttpMethod;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.client.RestTemplate;
public class Service1Application {
    public static void main(String[] args) {
        SpringApplication.run(Service1Application.class, args);
class Service1Controller {

    private static final Logger LOG = Logger.getLogger(Service1Controller.class.getName());
    RestTemplate restTemplate;
    public RestTemplate getRestTemplate() {
        return new RestTemplate();
    public Sampler alwaysSampler() {
        return Sampler.ALWAYS_SAMPLE;
    public String service1() 
        LOG.info("Inside Service 1..");         
String response = (String)   restTemplate.exchange("http://localhost:8082/service2", HttpMethod.GET, null, new ParameterizedTypeReference<String>() {}).getBody();
        return response;

Appication Configuration

As all services will run in a single machine, so we need to run them in different ports. Also to identify in Zipkin, we need to give proper names. so configure the application name and port information in application.properties file under the resources folder.

application.propertiesserver.port = 8081
spring.application.name = zipkin-server1

Similarly, for the other 2 services, we will use ports 8082, 8083 and their name will also be zipkin-server2 and zipkin-server3

Also, we have intentionally introduced a delay in the second service so that we can view that in Zipkin.

Above project is available in below github location

Github repo : https://github.com/mvtechbytes/Zipkin-Sleuth

On running app using bat files

Find Traces
Individual Trace
Trace details


Different ways of sorting an User Object

There are many ways to sort a java object but it is very hard to figure out which one is more efficient. Here is an example which describes different ways of executing sorting mechanism for User object based on age.

Try to run this application in you local machine to see which method is more efficient and good to use in our regular programming life.

package com.malliktalksjava.java8;

import java.util.ArrayList;
import java.util.Collections;
import java.util.Comparator;
import java.util.List;

import static java.util.stream.Collectors.toList;

public class SortingExampleUser {

    public static void main(String[] args) {
        List<User> userList = new ArrayList<>();
        userList.add(new User("Ram", 28));
        userList.add(new User("Raj", 35));
        userList.add(new User("Rakesh", 31));
        userList.add(new User("Peter", 30));
        userList.add(new User("John", 25));
        userList.add(new User("Sri", 55));

        long starttime = System.currentTimeMillis();
        System.out.println("sortListUsingCollections : " + sortListUsingCollections(userList));
        System.out.println("Time Taken in Millis : " + (System.currentTimeMillis() - starttime));

        long starttime2 = System.currentTimeMillis();
        System.out.println("sortListUsingCollections : " + sortListUsingStreams(userList));
        System.out.println("Time Taken in Millis  2: " + (System.currentTimeMillis() - starttime2));

        long starttime3 = System.currentTimeMillis();
        System.out.println("sortListUsingCollections : " + sortUsingLambda(userList));
        System.out.println("Time Taken in Millis  2: " + (System.currentTimeMillis() - starttime3));


    //using Collections.sort
    private static List<User> sortListUsingCollections(List<User> list){

        Collections.sort(list, Comparator.comparingInt(User::getAge));
        //Collections.reverse(list); // un comment if for descending order

        return list;

    //using streams and comparator
    private static List<User> sortListUsingStreams(List<User> list){

        return list.stream()
                //.sorted(Comparator.comparingInt(User::getAge).reversed()) //-- for reverse order uncomment this line and comment above line

    //using lambda expressions
    private static List<User> sortUsingLambda(List<User> list){

        return list.stream()
                .sorted((User user1, User user2) -> user1.getAge() > user2.getAge() ? 1: 0)
                //.sorted((User user1, User user2) -> user1.getAge() < user2.getAge() ? 1: 0) - uncomment if reverse order needed


class User{
    private String name;
    private int age;

    public User(String name, int age) {
        this.name = name;
        this.age = age;

    public String getName() {
        return name;

    public void setName(String name) {
        this.name = name;

    public int getAge() {
        return age;

    public void setAge(int age) {
        this.age = age;

    public String toString() {
        return "User{" +
                "name='" + name + '\'' +
                ", age=" + age +

Selenium Automation – Open URL in multiple browsers

This example shows how to open a url in multiple browsers for browser based testing using Selenium and WebdriverManager.

package seleniumProjects;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.edge.EdgeDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.safari.SafariDriver;

import io.github.bonigarcia.wdm.WebDriverManager;
import io.github.bonigarcia.wdm.config.DriverManagerType;

public class StartBrowser{
	static WebDriver driver = null;
	static String[] appList= {"chrome","firefox","edgedriver", "safari"};
	public static void main(String[] args) throws Exception {
		for(int i=0;i<appList.length;i++) {

	public static void browserStart(String appName, String appUrl)
			throws InstantiationException, IllegalAccessException {

		if (appName.equals("chrome")) { //Run Chrome browser
			driver = new ChromeDriver();
		} else if (appName.equals("firefox")) { //Run in Firefox broweser
			driver = new FirefoxDriver();
		} else if (appName.equals("edgedriver")) { // Run in Edge browser
			driver = new EdgeDriver();
		} else if (appName.equals("safari")) { //Run in Safari browser
            //For Safari browser, you need enable 
			//'Allow Remote Automation' under develop menu
			DriverManagerType safari = DriverManagerType.SAFARI;
			driver = new SafariDriver();



	public static void browserClose() {

Apache FreeMarker for transformation between data formats

In this post, we will learn how to use Apache FreeMarker for data format transformations

What is Apache FreeMarker?

Apache FreeMarker is a template engine: a Java library to generate text output (HTML web pages, e-mails, configuration files, XML, JSON, source code, etc.) based on templates and changing data. Templates are written in the FreeMarker Template Language (FTL), which is a simple, specialized language (not a full-blown programming language like PHP). Usually, a general-purpose programming language (like Java) is used to prepare the data (issue database queries, do business calculations). Then, Apache FreeMarker displays that prepared data using templates. In the template you are focusing on how to present the data, and outside the template you are focusing on what data to present.

If your project needs you to transform between data formats like XML to JSON or vice versa. Such transformations can be accomplished using FreeMarker

Apache FreeMarker for Data Transformations

XML TO JSON Transformation using FreeMarker

We will use SpringBoot project created using Spring Initilizer. https://start.spring.io/

FreeMarker Transformations – Project Structure

Firstly add dependencies to pom.xml




Add XML to transform in src/main/resources folder – test.xml

<?xml version="1.0" encoding="UTF-8"?>

Add FTL Template in src/main/resources/templates folder – FTL file: xml2json.ftl

<#assign data = xml['child::node()']>
    "employee": {
        "id": ${data.employee.id},
        "name": "${data.employee.name}",
        "location": "${data.employee.location}"

Create FmtManager to load and process template as below

package com.mvtechbytes.fmt;

import java.io.IOException;
import java.io.StringWriter;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Map;

import freemarker.cache.StringTemplateLoader;
import freemarker.ext.beans.BeansWrapperBuilder;
import freemarker.template.Configuration;
import freemarker.template.Template;

public class FmtManager {

    private Configuration freemarkerConfig;
    private static final String TEMPLATE_DIRECTORY = "src/main/resources/templates/";

    public FmtManager() {
        freemarkerConfig = new Configuration(Configuration.VERSION_2_3_23);
        freemarkerConfig.setObjectWrapper(new BeansWrapperBuilder(Configuration.VERSION_2_3_23).build());
        freemarkerConfig.setTemplateLoader(new StringTemplateLoader());

    private Template loadTemplate(String templateName, String templatePath) {
        try {
            String templateContent = new String(Files.readAllBytes(Paths.get(templatePath)));
            ((StringTemplateLoader) freemarkerConfig.getTemplateLoader()).putTemplate(templateName, templateContent);
            return freemarkerConfig.getTemplate(templateName);
        } catch (IOException e) {
            throw new RuntimeException(e);

    public String processTemplate(String templateName, Map<String, Object> data) {
        Template template = loadTemplate(templateName, TEMPLATE_DIRECTORY + templateName + ".ftl");
        try (StringWriter writer = new StringWriter()) {
            template.process(data, writer);
            return writer.toString();
        } catch (Exception e) {
            throw new RuntimeException(e);

After adding all the code in the respective folders. Execution of use case can be done using below static method with FmtManager bean injected

public static void xmlToJson(FmtManager templateManager) throws Exception {

        String xmlString = new String(Files.readAllBytes(Paths.get("src/main/resources/test.xml")));
        NodeModel xmlNodeModel = NodeModel.parse(new InputSource(new StringReader(xmlString)));

        Map<String, Object> data = new HashMap<>();
        data.put("xml", xmlNodeModel);

        String json = templateManager.processTemplate("xml2json", data);


Execution Log Output:

12:48:44.926 [main] DEBUG freemarker.cache - TemplateLoader.findTemplateSource("xml2json"): Found
12:48:44.929 [main] DEBUG freemarker.cache - Loading template for "xml2json"("en_US", UTF-8, parsed) from "xml2json"
"employee": {
"id": 101,
"name": "Vikas",
"location": "Toronto"

JSONTOXML Transformation using FreeMarker

Add JSON to transform in src/main/resources folder – test.json

  "data": {
    "employee": {
      "empid": 2012,
      "empname": "Virat",
      "location": "Hyderabad"

Add FTL Template in src/main/resources/templates folder – FTL file: json2xml.ftl

<#-- @ftlvariable name="JsonUtil" type="de.consol.jbl.util.JsonUtil" -->
<#assign body = JsonUtil.jsonToMap(input)>
<?xml version="1.0" encoding="UTF-8"?>

Create FmtJSONUtil – This to convert json to Java object

package com.mvtechbytes.fmt;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;

public class FmtJsonUtil {
    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();

    public static Map<String, Object> jsonToMap(String json) throws IOException {
        return OBJECT_MAPPER.readValue(json, new TypeReference<HashMap<String, Object>>(){});

After adding all the code in the respective folders. Execution of use case can be done using below static method with FmtManager bean injected

private static void jsonToXml(FmtManager templateManager) throws IOException, TemplateModelException {
		 String input = new String(Files.readAllBytes(Paths.get("src/main/resources/test.json")));

	        Map<String, Object> data = new HashMap<>();
	        data.put("input", input);

	        TemplateHashModel staticModels = new BeansWrapperBuilder(Configuration.VERSION_2_3_23).build().getStaticModels();
	        data.put("JsonUtil", staticModels.get(FmtJsonUtil.class.getName()));

	        String output = templateManager.processTemplate("json2xml", data);


Execution Log Output:

<?xml version="1.0" encoding="UTF-8"?>

Full sourcecode is available in below github link



REST API Components – Standards and Design aspects

In this post, we will see the different REST API components w.r.t standards and design aspects

Query parameters and QueryString length in HTTP GET

Security Aspect:

Although officially there is no limit specified by RFC 2616, many security protocols and recommendations state that maxQueryStrings on a server should be set to a maximum character limit of 1024. While the entire URL, including the querystring, should be set to a max of 2048 characters. This is to prevent the Slow HTTP Request DDOS vulnerability on a web server. This typically shows up as a vulnerability on the Qualys Web Application Scanner and other security scanners.

Please see the below example code for Windows IIS Servers with Web.config:

        <requestLimits maxQueryString="1024" maxUrl="2048">
              <add header="Content-type" sizeLimit="100" />

This would also work on a server level using machine.config.

Note: Limiting query string and URL length may not completely prevent Slow HTTP Requests DDOS attack but it is one step you can take to prevent it.

414 URI Too Long (RFC 7231):

The URI provided was too long for the server to process. Often the result of too much data being encoded as a query-string of a GET request, in which case it should be converted to a POST request. Called “Request-URI Too Long” previously.

Browser restrictions:

  • Microsoft Internet Explorer (Browser)
    Microsoft states that the maximum length of a URL in Internet Explorer is 2,083 characters, with no more than 2,048 characters in the path portion of the URL. Attempts to use URLs longer than this produced a clear error message in Internet Explorer.
  • Microsoft Edge (Browser)
    The limit appears to be around 81578 characters.
  • Chrome
    It stops displayingthe URL after 64k characters, but can serve more than 100k characters. No further testing was done beyond that.
  • Firefox (Browser)
    After 65,536 characters, the location bar no longer displays the URL in Windows Firefox 1.5.x. However, longer URLs will work. No further testing was done after 100,000 characters.
  • Safari (Browser)
    At least 80,000 characters will work. Testing was not tried beyond that.
  • Opera (Browser)
    At least 190,000 characters will work. Stopped testing after 190,000 characters. Opera 9 for Windows continued to display a fully editable, copyable and pasteable URL in the location bar even at 190,000 characters.
  • Apache (Server)
    Early attempts to measure the maximum URL length in web browsers bumped into a server URL length limit of approximately 4,000 characters, after which Apache produces a “413 Entity Too Large” error. The current up to date Apache build found in Red Hat Enterprise Linux 4 was used. The official Apache documentation only mentions an 8,192-byte limit on an individual field in a request.
  • Microsoft Internet Information Server (Server)
    The default limit is 16,384 characters (yes, Microsoft’s web server accepts longer URLs than Microsoft’s web browser). This is configurable.
  • Perl HTTP::Daemon (Server)
    Up to 8,000 bytes will work. Those constructing web application servers with Perl’s HTTP::Daemon module will encounter a 16,384 byte limit on the combined size of all HTTP request headers. This does not include POST-method form data, file uploads, etc., but it does include the URL. In practice this resulted in a 413 error when a URL was significantly longer than 8,000 characters. This limitation can be easily removed. Look for all occurrences of 16×1024 in Daemon.pm and replace them with a larger value. Of course, this does increase your exposure to denial of service attacks.

When to use @QueryParam versus @PathParam

REST may not be a standard as such, Most APIs tend to only have resource names and resource IDs in the path. Such as:


Some REST APIs use query strings for filtering, pagination and sorting, but REST isn’t a strict standard.

Recommendation is put any required parameters in the path, and any optional parameters should certainly be query string parameters. Putting optional parameters in the path will end up getting really messy when trying to write URL handlers that match different combinations.

When to use Headers versus URL parameters (PathParam or QueryParam)

GET /orders/view
(custom HTTP header) CLIENT_ID: 23

instead of

GET /orders/view/client_id/23 or
GET /orders/view/?client_id=23

The URL indicates the resource itself. A “client” is a resource that can be acted upon, so should be part of the base url: /orders/view/client/23.

Parameters are just that, to parameterize access to the resource. This especially comes into play with posts and searches: /orders/find?q=blahblah&sort=foo. There’s a fine line between parameters and sub-resources: /orders/view/client/23/active versus /orders/view/client/23?show=active. Recommendation is the sub-resource style and reserve parameters for searches.

Since each endpoint Represents a State Transfer (to mangle the mnemonic), custom headers should only be used for things that don’t involve the name of the resource (the url), the state of the resource (the body), or parameters directly affecting the resource (parameters). That leaves true metadata about the request for custom headers.

HTTP has a very wide selection of headers that cover most everything you’ll need. Where we could see custom headers which come up in a system to system request operating on behalf of a user. The proxy system will validate the user and add “X-User: userid” to the headers and use the system credentials to hit the endpoint. The receiving system validates that the system credentials are authorized to act on behalf of the user, then validate that the user is authorized to perform the action.

Custom headers have the following advantages:

  • Can be read easily by network tools/scripts (authentication, meta info)
  • Keeps urls free from security stuff (safer, not in browser/proxy caches)
  • Keeps urls cleaner: allows for better caching of resources