Connector to Java Client Library Example

This example walks you through the implementation of an Anypoint Connector to a service for which you have a Java client library. Follow the process outlined in this document to build a Java client library-based connector for any service.

Java client libraries are in many cases the best option for integrating with a remote service. If the client library is officially supported by the application provider, or even if it is unofficial but widely used, it likely implements best practices for integrating with the application in common use cases.

Assumptions

This document assumes:

Familiarity with Anypoint connector architecture as described in Anypoint Connector Concepts
Familiarity with the @Connector class methods of implementing authentication, as described in Authentication and Connection Management
If you are developing a client library-based connector, start by creating a new connector project, as described in Creating a Connector Project.

Client Library-Based Connector Architecture

The components of the connector include:

The Java client library that exposes the authentication methods and the supported operations
Supporting data model entity classes, which typically are defined by the Java client library
The @Connector class that you implement based on the DevKit-generated skeleton

In practice, the Java client library usually communicates with a RESTful or SOAP-based web service, but encapsulates all the client logic required in Java, and sidesteps the added work and complexity of building the client layer of the connector. The client library may also hide non-standard behavior of badly designed Web services behind a more manageable Java facade.

The Example Connector

The example for this discussion is the MuleSoft Twitter connector, which is based on the unofficial but widely used twitter4j client library. The source code for the Twitter Connector on GitHub, linenums is available and is referenced in the discussions.

Note that the Twitter connector illustrates many aspects of DevKit functionality added in the 3.5 releases of Mule, including DataSense Query Language support, OAuth 2 authentication, and paging results from operations. The focus in this discussion will be on the relationship between the client library and the @Connector class, which is effectively the same in DevKit 3.4 and 3.5 releases.

For authentication, the Twitter connector implements an interesting hybrid, defined in in TwitterConnector.java:

The Twitter4J client library implements its own OAuth support, which the Mule connector leverages.
Because the connector does not use DevKit’s OAuth support, it is possible to use DevKit’s connection management framework.

Adding the Client Library to the Connector Project

Depending upon how your client library is delivered, you can add it to your project in one of several ways. For example:

If the library is available via Maven, you can add it to your project POM file. For example, Twitter4J can be added as a Maven dependency from the central Maven repository as follows:
```
<dependency>
<groupId>org.twitter4j</groupId>
    <artifactId>twitter4j-core</artifactId>
    <version>[3.0,)</version>
</dependency>
```
You can simply download the JAR file and add it to the build path in Eclipse or IntelliJ.
If the source code is available, you can simply incorporate it into your project.

In the case of the Mule Twitter connector, in the current version the source code for Twitter4J is actually included in the connector directly. The Mule connector version of the Twitter4J library has been altered slightly, and the Mule connector accesses some internal classes, and customizes at least one:

import twitter4j.internal.http.alternative.HttpClientHiddenConstructionArgument;
import twitter4j.internal.http.alternative.MuleHttpClient;

For unofficial or unsupported client libraries, or any open source client library, this approach has the advantage that you have direct control of the contents of the client library, which you can tailor to work with Mule or to otherwise better fit your use case. (Of course, the tradeoff is that you then have to maintain the library going forward.)

Defining the @Connector Class

The first step is building out the @Connector class basic functionality around connection management and authentication.

Defining Entity Classes and Exceptions

In general, it is recommended that you define at least two exceptions for your connector: one to indicate connection and authentication-related failures, another to indicate all other failures, including, for example, passing invalid arguments to an operation. A separate exceptions package is a good place to define these.

In the case of the Twitter connector, the exceptions defined by Twitter4J are simply passed directly up to Mule.

Implementing Authentication and Connection Management

Authentication is implemented both in the @Connector class and in the client library. Details of implementing authentication in the @Connector class will depend upon the chosen authentication scheme, and on the client library support for authentication.

For OAuth, the @Connector class OAuth-related functionality may be used to collect required keys and secrets, or this may be managed by the client library by some other means.

For non-OAuth authentication, add @Configurable properties on the @Connector class, as described in Defining Configurable Connector Attributes, and take advantage of the connection management framework, as described in Authentication and Connection Management.

The API for the client library will expose methods or method parameters by which you pass those details to the library, either at initialization time or when calling operations. Details will vary by client library.

For details on the different schemes available and DevKit support for them, see:

Authentication and Connection Management for an overview
Implementing Connection Management for non-OAuth authentication schemes
Implementing OAuth 1.0 Authentication and Implementing OAuth 2.0 Authentication for OAuth-enabled schemes.

Leveraging Twitter4J OAuth Support

The Twitter connector implements an interesting hybrid, defined in in TwitterConnector.java:

The Twitter4J client library implements its own OAuth support, which the Mule connector leverages.
Because the connector does not use DevKit’s OAuth support, it is possible to use DevKit’s connection management framework.

Thus, we have the class definition with no @OAuth annotation:

@Connector(name = "twitter", schemaVersion = "2.4", description = "Twitter Integration", friendlyName = "Twitter",
minMuleVersion = "3.5", connectivityTesting = ConnectivityTesting.DISABLED)
public class TwitterConnector implements MuleContextAware {...

And a @Connect method with a @ConnectionKey set to the OAuth accessKey, and the usual associated @Disconnect, @ValidateConnection and @ConnectionIdentifier methods:

@Connect
    public void connect(@ConnectionKey String accessKey, String accessSecret) throws ConnectionException{
        ConfigurationBuilder cb = new ConfigurationBuilder();
        cb.setUseSSL(useSSL);
        cb.setHttpProxyHost(proxyHost);
        cb.setHttpProxyPort(proxyPort);
        cb.setHttpProxyUser(proxyUsername);
        cb.setHttpProxyPassword(proxyPassword);

        HttpClientHiddenConstructionArgument.setUseMule(true);
        twitter = new TwitterFactory(cb.build()).getInstance();


        twitter.setOAuthConsumer(consumerKey, consumerSecret);
        if (accessKey != null) {
            twitter.setOAuthAccessToken(new AccessToken(accessKey, accessSecret));
            setAccessToken(accessKey);
            setAccessTokenSecret(accessSecret);
        }
    }
...

    @Disconnect
    public void disconnect() {
        twitter = null;
    }

    @ValidateConnection
    public boolean validateConnection() {
        return twitter != null;
    }

    @ConnectionIdentifier
    public String getConnectionIdentifier() {
        return getAccessToken() + "-" + getAccessTokenSecret();
    }

On the other hand, we have a series of @Processor methods that implement OAuth-related functionality, like getting and managing an access token by calling functions exposed by class twitter4j.Twitter :

    /**
     * Set the OAuth verifier after it has been retrieved via requestAuthorization.
     * The resulting access tokens will be logged to the INFO level so the user can
     * reuse them as part of the configuration in the future if desired.
     * <p/>
     * {@sample.xml ../../../doc/twitter-connector.xml.sample twitter:setOauthVerifier}
     *
     *
     * @param requestToken request token from Twitter
     * @param oauthVerifier The OAuth verifier code from Twitter.
     * @return Twitter AccessToken info.
     * @throws TwitterException when Twitter service or network is unavailable
     */
    @Processor
    public AccessToken setOauthVerifier(@Optional RequestToken requestToken, String oauthVerifier) throws TwitterException {
        AccessToken accessToken;
        if (requestToken != null) {
            accessToken = twitter.getOAuthAccessToken(requestToken, oauthVerifier);
        }
        else {
            accessToken = twitter.getOAuthAccessToken(oauthVerifier);
        }


        logger.info("Got OAuth access tokens. Access token:" + accessToken.getToken()
                + " Access token secret:" + accessToken.getTokenSecret());


        return accessToken;
    }


  /**
   * Start the OAuth request authorization process.
   */


@Processor
    public RequestToken requestAuthorization(@Optional String callbackUrl) throws TwitterException {
        RequestToken token = twitter.getOAuthRequestToken(callbackUrl);
        return token;
    }


...
   public String getAccessToken() {
        return accessToken;
    }
    public void setAccessToken(String accessToken) {
        this.accessToken = accessToken;
    }

    public String getAccessTokenSecret() {
        return accessTokenSecret;
    }

    public void setAccessTokenSecret(String accessTokenSecret) {
        this.accessTokenSecret = accessTokenSecret;
    }

And the @Processor methods that actually call Twitter operations do not use the @OAuthProtected annotation:

@Processor
    public User showUser() throws TwitterException {
        return twitter.showUser(twitter.getId());
    }

You can dig into this code and use a similar implementation pattern if you are working with a client library that provides its own OAuth support.

Adding an Operation to the @Connector Class

At this point you can start adding operations to the connector.

With a client library, the steps to add an operation include:

Importing any Java entity classes defined by the client library, that are used as parameters or return value by the operation, as well as any exceptions the client library may raise.
Adding a @Processor method on the @Connector class, that calls an operation on the client instance.

Depending upon your specific client class, you may need to add authentication functionality in the operation methods to handle authentication.

Apply a Test-Driven Approach

Based on MuleSoft experience, most successful connector implementation projects follow a cycle similar to test-driven development when building out operations on a connector:

Determine detailed requirements for the operation-- entities (POJOs or Maps with specific content) it can accept as inputs or return as responses, any edge cases like invalid values, values of the wrong type, and so on, and what exceptions the operation may raise.
Implement JUnit tests that cover those requirements;
Implement enough of your operation to pass those tests, including creating new entity classes and exceptions;
Update your @Connector class and other code with the comments that populate the Javadoc related to the operation

Iterate until you cover all the scenarios covered in your requirements for a given operation. Then use the same cycle to implement each operation, until your connector functionality is complete.

If your client library is well-documented, the expected behaviors for operations should be clear, and you may be able to get away with less unit testing for edge cases and certain exceptional situatons – but bear in mind that your connector is only as reliable as the Java client you based it on.

You may ask, "When do I try my connector in Studio?" It is useful, as well as gratifying, to manually test each operation as you go, in addition to the automated JUnit tests:

You get to see basic operation functionality in action as you work on it, which gives you a sense of progress
You get to see how the connector appears in the Studio UI, something the automated unit tests cannot show you. For example, text from the Javadoc comments is used to populate tooltips for the fields in the dialog boxes in the connector.

This will provide the opportunity to polish the appearance of the connector, improve the experience with sensible defaults, and so on.

However, this does not take away from the value of the test-driven approach. Many connector development projects have bogged down or have produced hard-to-use connectors because of a failure to define tests as you define the operations – it seems like (and is) more work up front, but it does pay off – you get to a better result, faster.

Implementing Operations

The Twitter connector implements a rich set of operations; some of the simpler ones are shown below.

/**
     * Returns a single status, specified by the id parameter below. The status's
     * author will be returned inline. <br>
     * This method calls http://api.twitter.com/1.1/statuses/show
     * <p/>
     * {@sample.xml ../../../doc/twitter-connector.xml.sample twitter:showStatus}
     *
     * @param id the numerical ID of the status you're trying to retrieve
     * @return a single {@link Status}
     * @throws twitter4j.TwitterException when Twitter service or network is unavailable
     * @see <a href="http://dev.twitter.com/doc/get/statuses/show/:id">GET
     *      statuses/show/:id | dev.twitter.com</a>
     */
    @Processor
    public Status showStatus(long id) throws TwitterException {
        return twitter.showStatus(id);
    }


    /**
     * Answers user information for the authenticated user
     * <p/>
     * {@sample.xml ../../../doc/twitter-connector.xml.sample twitter:showUser}
     *
     * @return a {@link User} object
     * @throws TwitterException when Twitter service or network is unavailable
     */
    @Processor
    public User showUser() throws TwitterException {
        return twitter.showUser(twitter.getId());
    }


    /**
     * Search for places that can be attached to a statuses/update. Given a latitude
     * and a longitude pair, or and IP address, this request will return a list of
     * all the valid places that can be used as the place_id when updating a status.
     * <p/>
     * {@sample.xml ../../../doc/twitter-connector.xml.sample twitter:searchPlaces}
     *
     * @param latitude  latitude coordinate. Mandatory if ip is not specified
     * @param longitude longitude coordinate.
     * @param ip        the ip. Mandatory if coordinates are not specified
     * @return a {@link ResponseList} of {@link Place}
     * @throws TwitterException when Twitter service or network is unavailable
     */
    @Processor
    public ResponseList<Place> searchPlaces(@Placement(group = "Coordinates") @Optional Double latitude,
                                            @Placement(group = "Coordinates") @Optional Double longitude,
                                            @Optional String ip) throws TwitterException {
        return twitter.searchPlaces(createQuery(latitude, longitude, ip));
    }


    private GeoQuery createQuery(Double latitude, Double longitude, String ip) {
        if (ip == null) {
            return new GeoQuery(new GeoLocation(latitude, longitude));
        }
        return new GeoQuery(ip);
    }

Things to note:

All of these operations call methods on the client instance stored in the twitter property.
Annotations like @Optional, @Default, and @Placement are widely used to improve the configuration behavior of the connector and its appearance in Studio.
Because the authentication is all handled by the Java client and a few methods in the @Connector class noted above, no authentication-related code is included in the @Processor methods.

Creating JavaDoc and Samples for Operations

The JavaDoc for each operation includes a pointer to the sample code file ../../../doc/twitter-connector.xml.sample, as well as the usual @param and @return comments. DevKit will enforce the inclusion of these code samples, as well as the usual DevKit and will check the samples you provide against the parameters defined for those operations. See Creating DevKit Connector Documentation for details on creating the required documentation for each of your operations.

Creating Unit Tests for Operations

For each operation, as you define it, you should create the unit tests that exercise it. The generated project skeleton created by the DevKit Maven archetype includes a unit test suite directory under ./src/test. DevKit defines a unit test framework based on JUnit.

For details on creating unit tests, see Developing DevKit Connector Tests.

Gartner names MuleSoft a Leader

Unleash the power of Salesforce Customer 360 through integration

Anypoint Platform Fundamentals

MuleSoft at World Tour