Apache FreeMarker for transformation between data formats


In this post, we will learn how to use Apache FreeMarker for data format transformations

What is Apache FreeMarker?

Apache FreeMarker is a template engine: a Java library to generate text output (HTML web pages, e-mails, configuration files, XML, JSON, source code, etc.) based on templates and changing data. Templates are written in the FreeMarker Template Language (FTL), which is a simple, specialized language (not a full-blown programming language like PHP). Usually, a general-purpose programming language (like Java) is used to prepare the data (issue database queries, do business calculations). Then, Apache FreeMarker displays that prepared data using templates. In the template you are focusing on how to present the data, and outside the template you are focusing on what data to present.

If your project needs you to transform between data formats like XML to JSON or vice versa. Such transformations can be accomplished using FreeMarker

Apache FreeMarker for Data Transformations

XML TO JSON Transformation using FreeMarker

We will use SpringBoot project created using Spring Initilizer. https://start.spring.io/

FreeMarker Transformations – Project Structure

Firstly add dependencies to pom.xml

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-freemarker</artifactId>
    </dependency>
    <dependency>
        <groupId>com.googlecode.json-simple</groupId>
        <artifactId>json-simple</artifactId>
        <version>1.1</version>
    </dependency>

    <dependency>
        <groupId>no.api.freemarker</groupId>
        <artifactId>freemarker-java8</artifactId>
        <version>1.3.0</version>
    </dependency>

    <dependency>
        <groupId>org.everit.json</groupId>
        <artifactId>org.everit.json.schema</artifactId>
        <version>1.5.1</version>
    </dependency>

Add XML to transform in src/main/resources folder – test.xml

<?xml version="1.0" encoding="UTF-8"?>
<data>
    <employee>
        <id>30123</id>
        <name>Ben</name>
        <location>Toronto</location>
    </employee>
</data>

Add FTL Template in src/main/resources/templates folder – FTL file: xml2json.ftl

<#assign data = xml['child::node()']>
{
    "employee": {
        "id": ${data.employee.id},
        "name": "${data.employee.name}",
        "location": "${data.employee.location}"
    }
}

Create FmtManager to load and process template as below

package com.mvtechbytes.fmt;

import java.io.IOException;
import java.io.StringWriter;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Map;

import freemarker.cache.StringTemplateLoader;
import freemarker.ext.beans.BeansWrapperBuilder;
import freemarker.template.Configuration;
import freemarker.template.Template;

public class FmtManager {

    private Configuration freemarkerConfig;
    private static final String TEMPLATE_DIRECTORY = "src/main/resources/templates/";

    public FmtManager() {
        freemarkerConfig = new Configuration(Configuration.VERSION_2_3_23);
        freemarkerConfig.setTagSyntax(Configuration.ANGLE_BRACKET_TAG_SYNTAX);
        freemarkerConfig.setDefaultEncoding("UTF-8");
        freemarkerConfig.setNumberFormat("computer");
        freemarkerConfig.setObjectWrapper(new BeansWrapperBuilder(Configuration.VERSION_2_3_23).build());
        freemarkerConfig.setTemplateLoader(new StringTemplateLoader());
    }

    private Template loadTemplate(String templateName, String templatePath) {
        try {
            String templateContent = new String(Files.readAllBytes(Paths.get(templatePath)));
            ((StringTemplateLoader) freemarkerConfig.getTemplateLoader()).putTemplate(templateName, templateContent);
            return freemarkerConfig.getTemplate(templateName);
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    public String processTemplate(String templateName, Map<String, Object> data) {
        Template template = loadTemplate(templateName, TEMPLATE_DIRECTORY + templateName + ".ftl");
        try (StringWriter writer = new StringWriter()) {
            template.process(data, writer);
            return writer.toString();
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }
}

After adding all the code in the respective folders. Execution of use case can be done using below static method with FmtManager bean injected

public static void xmlToJson(FmtManager templateManager) throws Exception {

        String xmlString = new String(Files.readAllBytes(Paths.get("src/main/resources/test.xml")));
        NodeModel xmlNodeModel = NodeModel.parse(new InputSource(new StringReader(xmlString)));

        Map<String, Object> data = new HashMap<>();
        data.put("xml", xmlNodeModel);

        String json = templateManager.processTemplate("xml2json", data);

        System.out.println(json);
 }

Execution Log Output:

12:48:44.926 [main] DEBUG freemarker.cache - TemplateLoader.findTemplateSource("xml2json"): Found
12:48:44.929 [main] DEBUG freemarker.cache - Loading template for "xml2json"("en_US", UTF-8, parsed) from "xml2json"
{
"employee": {
"id": 101,
"name": "Vikas",
"location": "Toronto"
}
}

JSONTOXML Transformation using FreeMarker

Add JSON to transform in src/main/resources folder – test.json

{
  "data": {
    "employee": {
      "empid": 2012,
      "empname": "Virat",
      "location": "Hyderabad"
    }
  }
}

Add FTL Template in src/main/resources/templates folder – FTL file: json2xml.ftl

<#-- @ftlvariable name="JsonUtil" type="de.consol.jbl.util.JsonUtil" -->
<#assign body = JsonUtil.jsonToMap(input)>
<?xml version="1.0" encoding="UTF-8"?>
<data>
    <employee>
        <id>${body.data.employee.empid}</id>
        <name>${body.data.employee.empname}</name>
     	<location>${body.data.employee.location}</location>
    </employee>
</data>

Create FmtJSONUtil – This to convert json to Java object

package com.mvtechbytes.fmt;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;

public class FmtJsonUtil {
    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();

    public static Map<String, Object> jsonToMap(String json) throws IOException {
        return OBJECT_MAPPER.readValue(json, new TypeReference<HashMap<String, Object>>(){});
    }
}

After adding all the code in the respective folders. Execution of use case can be done using below static method with FmtManager bean injected

private static void jsonToXml(FmtManager templateManager) throws IOException, TemplateModelException {
		 String input = new String(Files.readAllBytes(Paths.get("src/main/resources/test.json")));

	        Map<String, Object> data = new HashMap<>();
	        data.put("input", input);

	        TemplateHashModel staticModels = new BeansWrapperBuilder(Configuration.VERSION_2_3_23).build().getStaticModels();
	        data.put("JsonUtil", staticModels.get(FmtJsonUtil.class.getName()));

	        String output = templateManager.processTemplate("json2xml", data);

	        System.out.println(output);
	}

Execution Log Output:

<?xml version="1.0" encoding="UTF-8"?>
<data>
    <employee>
        <id>2012</id>
        <name>Virat</name>
     	<location>Hyderabad</location>
    </employee>
</data>

Full sourcecode is available in below github link

https://github.com/malliktalksjava/FreeMarkerTransformations

References:

What are the differences between SOAP WS and RESTful WS?


SOAP Web Services RESTfull Web Services
The SOAP WS supports both remote procedure call (i.e. RPC) and message oriented middle-ware (MOM) integration styles. The Restful Web Service supports only RPC integration style.
The SOAP WS is transport protocol neutral. Supports multiple protocols like HTTP(S),  Messaging, TCP, UDP SMTP, etc. The REST is transport protocol specific. Supports only HTTP or HTTPS protocols.
The SOAP WS permits only XML data format. You define operations, which tunnels through the POST. The focus is on accessing the named operations and exposing the application logic as a service. The REST permits multiple data formats like XML, JSON data, text, HTML, etc. Any browser can be used because the REST approach uses the standard GET, PUT, POST, and DELETE Web operations. The focus is on accessing the named resources and exposing the data as a service. REST has AJAX support. It can use the XMLHttpRequest object. Good for stateless CRUD (Create, Read, Update, and Delete) operations.GET – represent()POST – acceptRepresention()

PUT – storeRepresention()

DELETE – removeRepresention()

SOAP based reads cannot be cached. REST based reads can be cached. Performs and scales better.
SOAP WS supports both SSL security and WS-security, which adds some enterprise security features like maintaining security right up to the point where it is needed, maintaining identities through intermediaries and not just point to point SSL only, securing different parts of the message with different security algorithms, etc. The REST supports only point-to-point SSL security. The SSL encrypts the whole message, whether all of it is sensitive or not.
The SOAP has comprehensive support for both ACID based transaction management for short-lived transactions and compensation based transaction management for long-running transactions. It also supports two-phase commit across distributed resources. The REST supports transactions, but it is neither ACID compliant nor can provide two phase commit across distributed transactional resources as it is limited by its HTTP protocol.
The SOAP has success or retries logic built in and provides end-to-end reliability even through SOAP intermediaries. REST does not have a standard messaging system, and expects clients invoking the service to deal with communication failures by retrying.

 

Other Useful links:

 Click here to know more about webservices

Click here to know more about RESTfull web services.

Click here for Web services Question and Answers.

Click here to know how to create a SOAP web service.

Click here to know how to write web service client suing java.

What are the benefits of XML?


There are many benefits of using XML on the Web:

  • Simplicity– Information coded in XML is easy to read and understand, plus it can be processed easily by computers.
  • Openness– XML is a W3C standard, endorsed by software industry market leaders.
  • Extensibility – There is no fixed set of tags. New tags can be created as they are needed.
  • Self-description– In traditional databases, data records require schemas set up by the database administrator. XML documents can be stored without such definitions, because they contain meta data in the form of tags and attributes.
  • Contains machine-readable context information- Tags, attributes and element structure provide context information that can be used to interpret the meaning of content, opening up new possibilities for highly efficient search engines, intelligent data mining, agents, etc.
  • Separates content from presentation– XML tags describe meaning not presentation. The motto of HTML is: “I know how it looks”, whereas the motto of XML is: “I know what it means, and you tell me how it should look.” The look and feel of an XML document can be controlled by XSL style sheets, allowing the look of a document to be changed without touching the content of the document. Multiple views or presentations of the same content are easily rendered.
  • Supports multilingual documents and Unicode-This is important for the internationalization of applications.
  • Facilitates the comparison and aggregation of data – The tree structure of XML documents allows documents to be compared and aggregated efficiently element by element.
  • Can embed multiple data types – XML documents can contain any possible data type – from multimedia data (image, sound and video) to active components (Java applets, ActiveX).
  • Can embed existing data – Mapping existing data structures like file systems or relational databases to XML is simple. XML supports multiple data formats and can cover all existing data structures.
  • Provides a ‘one-server view’ for distributed data – XML documents can consist of nested elements that are distributed over multiple remote servers. XML is currently the most sophisticated format for distributed data – the World Wide Web can be seen as one huge XML database.

WEB Services Interview Questions


1)What is web Services ?
Getting the services through the web.

2)What is JAX-RPC?
JAX-RPC means java api for xml based RPC is a java api for developing and using web-services.
An RPC based web-service is a collection of procedures that can be called by remote client over the internet.
Ex: A typical RPC based web service is a stock quote service that takes a SOAP request for the price of a specified stock and returns the price via SOAP.

3)Explain about Web Services
A Web service, a server application that implements the procedures that available for clients to call, deployed on server-side container.
A web service can make itself available to potential clients by describing itself in a Web Service Description Language (WSDL) document.

4)What is WSDL?
WSDL is an XML document that gives all pertinent information about web service, the operation that can be called on it, parameters for those operations, and the location of where to send requests.
A web client can use WSDL document to discover the what the service offers and how to access it.

5)Explain some of the features of the web services?
Some of the features of the web services are:
1.Interoperability
Using JAX-RPC, a client written in a language other than Java programming language can access a web service developed and deployed on the Java platform. Conversely,a client written in the Java programming language can communicate with a service that was developed and deployed using some other platform.
2.Ease of use:
Using JAX-RPC makes the user to feel comfortable over the RPC. The internal complicated infrastructure, like marshaling, unmarshaling, and transmission details..etc, will be taken care automatically.
3.Advanced features

6)What makes interoperability is possible?
JAX-RPC support for SOAP and WSDL.

7)What is JAXM?
Java API for Messaging provides a standard way to send XML documents over the Internet from the Java platform. It is based on SOAP 1.1 and SOAP with attachments specifications, which define a basic framework for exchanging XML messages.
8)The features of JAXM over the RPC including JAX-RPC
a) One way(asynchronous)messaging.
b) Routing of message to more than one party.
c) Reliable messaging features such as guarantied delivery.

9)What is JAXR?
Java API for XML registries provides a convenient way to access standard business registries over the Internet. Using JAXR one can register the business registry and one can search for a business registry for other business.

10) What tools do you use to test your Web Services?

 SoapUI tool for SOAP WS and the Firefox “poster” plugin for RESTFul services.

11) What is the difference between SOA and a Web service? 

SOA is a software design principle and an architectural pattern for implementing loosely coupled, reusable and coarse grained services. You can implement SOA using any protocols such as HTTP, HTTPS, JMS, SMTP, RMI, IIOP (i.e. EJB uses IIOP), RPC etc. Messages can be in XML or Data Transfer Objects (DTOs).

Web service is an implementation technology and one of the ways to implement SOA. You can build SOA based applications without using Web services – for example by using other traditional technologies like Java RMI, EJB, JMS based messaging, etc. But what Web services offer is the standards based  and platform-independent service via HTTP, XML, SOAP, WSDL and UDDI, thus allowing interoperability between heterogeneous technologies such as J2EE and .NET.