Target Objectives and Goals

The goal is to connect as a backend application to an Oracle NetSuite instance and read data from the API. This must be possible without any human user interaction, either time-triggered or event-based.

Possible Approaches

Oracle NetSuite offers two kind of technical API implementations: The SOAP Webservice API over HTTP and a HTTP REST API. Both have their advantages and disadvantages as described here [1]

SOAP Webservice API

A WSDL with several linked XSD schemas is accessible at the NetSuite server and can be used to generate client stubs in various technologies. With Java technology and the use of the SpringFramework it is pretty straightforward to generate all required types and client stub implementation within the build process (see below).

HTTP REST API

The provided REST API offers a variety to work with. Basically it is differentiated into a resource based REST Record Service and a query based REST Query Service. The first one provides CRUD operations on NetSuite records and is compliant to the RMM Level 3 including HAL links. The query service doesn't follow the REST philosophy at all, only works with one resource and one HTTP verb and is used to query/fetch data from NetSuite dynamically.

REST Record Service

With the REST Record Service actually two network calls are required to get the representation of a NetSuite record. The first call is used to filter the results whereas the result set just contains HAL links to the actual resource representations and metadata about the result. With a second call the representation can then be fetched based on the record id.

First call to filter the result set and get the references:

GET https://my-account.suitetalk.api.netsuite.com/services/rest/record/v1/assemblyItem

Result:

{
  "links": [
    {
      "rel": "self",
      "href": "https://my-account.suitetalk.api.netsuite.com/services/rest/record/v1/assemblyItem"
    }
  ],
  "count": 1,
  "hasMore": false,
  "items": [
    {
      "links": [
        {
          "rel": "self",
          "href": "https://my-account.suitetalk.api.netsuite.com/services/rest/record/v1/assemblyitem/751"
        }
      ],
      "id": "751"
    }
  ],
  "offset": 0,
  "totalResults": 1
}

In a second step we need to follow the HAL link to get the resource representation that also includes a lot of HAL links to secondary resources beside own properties. Embedded resources are not provided. Even so, the REST Record Service is a well designed and faithful REST API it is cumbersome to work with, in particular if aggregated data is needed.

REST Query Service

As mentioned, the REST Query Service is in that different, that it only supports HTTP POST to the /suiteql resource. All filtering and projection is done in a SuiteQL query (similar to SQL) in the request body. The request header requires the "Prefer: transient" attribute and a bearer token in the HTTP authorization header.

POST https://my-account.suitetalk.api.netsuite.com/services/rest/query/v1/suiteql

HEADER
> content-type: application/json
> prefer: transient
> authorization: Bearer <access-token>
> accept: */*

BODY
{
  "q": "SELECT * FROM item where fullname like '%Foo%'"
}

The response header and body may look like:

< HTTP/2 200 
< content-type: application/vnd.oracle.resource+json; type=collection; charset=UTF-8
< preference-applied: transient

Body:

{
  "links": [
    {
      "rel": "self",
      "href": "https://my-account.suitetalk.api.netsuite.com/services/rest/query/v1/suiteql"
    }
  ],
  "count": 1,
  "hasMore": false,
  "items": [
    {
      "links": [],
      "itemid": "Foo Bar",
      "itemtype": "Assembly",
      "lastmodifieddate": "12.9.2023",
      .. other properties here ..
    }
  ],
  "offset": 0,
  "totalResults": 1
}

Also the approach with the REST Query Service is not very rest-ish, it offers an unbeatable flexible way to query for information in a single fetch, similar to the use of a GraphQL API. No overfetching - no underfetching.

Solution

The OpenWMS.org WMS NetSuite Adapter uses the REST Query Service to query for projections on NetSuite records and the REST Record Service for write operations. The use of SOAP webservices is omitted for now.

Authentication and Authorization

Accessing the REST API requires some effort. API resources are generally protected against unauthorized access with either OAuth1 (called TBA - Token Based Authorization) or OAuth2. Since OAuth1 is not an option with new client applications, the only way to go is with OAuth2. From the typical OAuth2 authorization flows, only the Authorization Grant Flow and the Client Credentials Flow are supported by NetSuite. While the Authorization Code Flow is a frontend-legged flow and requires an user consent, the Client Credentials Flow is the right one to pick for backend applications connecting to NetSuite.

With the Client Credentials Flow, the client application usually only uses credentials (clientId and clientSecret) to authenticate and get the appropriate access token. More effort is required with NetSuite here.

User Setup

At first a human user with the proper set of permissions need to create a technical machine user account in NetSuite and a role. The new user must NOT be assigned to the Administrator role. The role has assigned a couple of permissions. Beside the permissions to access entity lists (like items) also general technical permissions must be granted to the new role as well. In this example the role for the webservice is called "REST Users" and the user "REST API User". Both entities can be created in the NetSuite UI application under "Setup > Users/Roles"

Client Setup

In a second step also the accessing client application must be registered in NetSuite.  A new application is registered under "Setup > Users/Roles > OAuth2 Authorized Applications". Restrict the authentication methods to only support the OAuth2 Client Credentials flow. The clientId and clientSecret credentials are directly displayed to the user after the client has been created. Copy and store both in a secured location for further usage.

Additionally a client certificate must be created and the public key must be uploaded to NetSuite. This certificate is used to authenticate the machine client later on. To create a client certificate key pair follow the instructions on the NetSuite documentation site. Navigate to the site "Setup > Integration > OAuth 2.0 Client Credentials Setup" and create a new configuration entry with the previously created API user (REST API User), the role (REST Users) and the entity (client application). Also the certificate must be uploaded here. This step is only required for the OAuth2 Client Credentials Flow used by backend applications. Frontend applications may use the OAuth2 Authorization Code Flow with user authentication interaction.

OAuth2 Client Credentials Authentication

After a client has been registered at the NetSuite instance it can authenticate itself with a signed JWT. So the (client) application must create a JWT and sign it with the previously created key. This is not enough. To ensure encrypted transport layer security the transport channel requires mTLS and the client must use the client certificate for that. The following steps are required:

The certificate must be stored in a Java keystore file in order to setup mTLS within a Java application:

keytool -import -keystore keystore.p12 -alias mycert -file auth-cert.pem -trustcacerts
OpenWMS.org NetSuite Adapter relies on SpringFramework and uses SpringBoot on top. For all the OAuth2 handling the spring-boot-starter-oauth2-client is used and needs some further customization.

Basically the SpringBoot OAuth2 Client support uses a RestTemplate internally to access the OAuth2 endpoints and retrieve tokens. The RestTemplate must be customized to support TLS and the client certificate:

@Bean RestTemplate secureRestTemplate(OWMSConfig owmsConfig) throws Exception {
    var socketFactory = new SSLConnectionSocketFactory(
            new SSLContextBuilder()
                    .loadTrustMaterial(
                            owmsConfig.getKeystore().getURL(), // classpath:keystore.p12
                            owmsConfig.getPassword(),
                            new TrustSelfSignedStrategy()
                    )
                    .setProtocol(owmsConfig.getProtocol()) // TLSv1.2
                    .build()
    );
    var restTemplate = new RestTemplate(
            new HttpComponentsClientHttpRequestFactory(
                    HttpClients.custom().setSSLSocketFactory(socketFactory).build()
            )
    );
    restTemplate.setMessageConverters(Arrays.asList(
            new FormHttpMessageConverter(),
            new OAuth2AccessTokenResponseHttpMessageConverter()
    ));
    restTemplate.setErrorHandler(new OAuth2ErrorResponseErrorHandler());
    return restTemplate;
}

As soon as mTLS is setup for the communication channel, the standard OAuth2 Client Credentials flow must be extended with additional request body attributes. As mentioned earlier NetSuite requires a self-signed JWT token as part of the Access Token retrieval process to identify the registered client and validate the JWS (signature) with the uploaded certificate. This signed JWT must be created by the application and renewed if necessary. It is sent as additional request body property (client_assertion) to the OAuth2 Access Token endpoint. Also the assertion type must be passed (fixed urn:ietf:params:oauth:client-assertion-type:jwt-bearer)

@Bean public AuthorizedClientServiceOAuth2AuthorizedClientManager authorizedClientServiceAndManager (
        ClientRegistrationRepository clientRegistrationRepository,
        OAuth2AuthorizedClientService authorizedClientService,
        @Qualifier("secureRestTemplate") RestTemplate restTemplate, NetSuiteProperties netSuiteProperties) {

    var accessTokenResponseClient = new DefaultClientCredentialsTokenResponseClient();
    var converter = new OAuth2ClientCredentialsGrantRequestEntityConverter();
    converter.addParametersConverter(new NetSuiteClientCredentialsEnhancer(new RequestTokenStore(netSuiteProperties)));
    accessTokenResponseClient.setRequestEntityConverter(converter);
    accessTokenResponseClient.setRestOperations(restTemplate);

    var authorizedClientProvider =
            OAuth2AuthorizedClientProviderBuilder.builder()
                    .clientCredentials(c -> c.accessTokenResponseClient(accessTokenResponseClient))
                    .build();
    var authorizedClientManager = new AuthorizedClientServiceOAuth2AuthorizedClientManager(
            clientRegistrationRepository, authorizedClientService);
    authorizedClientManager.setAuthorizedClientProvider(authorizedClientProvider);
    return authorizedClientManager;
}

The NetSuiteClientCredentialsEnhancer is a simple org.springframework.core.convert.converter.Converter that takes care to add the required properties to the request body. It uses a RequestTokenStore that is responsible to create, sign, validate and cache the NetSuite Request Token.

class NetSuiteClientCredentialsEnhancer implements Converter<OAuth2ClientCredentialsGrantRequest, MultiValueMap<String, String>> {

    ..

    @Override
    @Measured
    public MultiValueMap<String, String> convert(OAuth2ClientCredentialsGrantRequest source) {
        var parameters = new LinkedMultiValueMap<String, String>();
        parameters.add("client_assertion_type", "urn:ietf:params:oauth:client-assertion-type:jwt-bearer");
        parameters.add("client_assertion", requestTokenStore.getToken());
        return parameters;
    }
}

The RequestTokenStore uses the io.jsonwebtoken library to build the token.

public String getToken() {

    var now = System.currentTimeMillis() / 1000L;
    if (requestToken == null || requestToken.isExpired(now + 3600)) {
        var headers = new HashMap<String, Object>();
        headers.put(Header.TYPE, Header.JWT_TYPE);
        headers.put(ALGORITHM, SignatureAlgorithm.RS256.getValue());
        headers.put(KEY_ID, netSuiteProperties.getRequestToken().getKid());
        var token = Jwts.builder()
                .addClaims(
                    Jwts.claims(Map.of(
                            Claims.ISSUED_AT, now,
                            Claims.EXPIRATION, now + 3600,
                            "scope", new String[]{ netSuiteProperties.getRequestToken().getScope() },
                            Claims.AUDIENCE, netSuiteProperties.getRequestToken().getAudience(),
                            Claims.ISSUER, netSuiteProperties.getRequestToken().getIssuer()
                    ))
                )
                .setHeader(headers)
                .signWith(SignatureAlgorithm.RS256, signingKey).compact();
         this.requestToken = new SignedJwt(now, token);
    }
    return requestToken.getRequestToken();
}

Justification

After trying both alternatives, the SOAP webservices and the REST API, the decision has taken to solely use the REST API. One reason is because it is a modern fashioned way for connectivity over SOAP. Another reason against the SOAP API is the burden of client-side code generation that must be done. Apache Axis2, as recommended in the NetSuite examples, didn't seem to be the right and suitable choice for modern applications. Even the use of JAX-WS is painful to get all the proper and fitting dependencies right. Beside this, JAXB bindings had to be tweaked to get all the client stub classes generated. Finally a @WebServiceClient stub has been created to access the remote service. But after all, web service security hasn't been addressed so far.

/**
 * This class was generated by the JAX-WS RI.
 * JAX-WS RI 2.3.6
 * Generated source version: 2.2
 * 
 */
@WebServiceClient(name = "NetSuiteService",
  targetNamespace = "urn:platform_2023_1.webservices.netsuite.com",
  wsdlLocation = "https://webservices.netsuite.com/wsdl/v2023_1_0/netsuite.wsdl")
public class NetSuiteService
    extends Service
{ .. }

To sum up: Even with the OAuth2 obstacles, it was a way much easier to use the REST API. But only with the support of the REST Query Service. Only using the REST Record Service is not an option because of performance issues.

Alternatives

Use of SOAP Webservice

A Maven profile can be added to the project in order to do this with JAX-WS on demand:

<profiles>
    <profile>
        <id>generate</id>
        <properties>
            <jaxb2.version>2.5.0</jaxb2.version>
        </properties>
        <build>
            <plugins>
                <plugin>
                    <groupId>com.sun.xml.ws</groupId>
                    <artifactId>jaxws-maven-plugin</artifactId>
                    <version>2.3.6</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>wsimport</goal>
                            </goals>
                        </execution>
                    </executions>
                    <configuration>
                        <args>
                            <arg>-B-XautoNameResolution</arg>
                        </args>
                        <vmArgs>
                            <vmArg>-Djavax.xml.accessExternalSchema=all</vmArg>
                        </vmArgs>
                        <bindingFiles>
                            <bindingFile>${project.basedir}/src/jaxws/bindings.xjb</bindingFile>
                        </bindingFiles>
                        <packageName>org.openwms.wms.netsuite</packageName>
                        <wsdlUrls>
                            <wsdlUrl>https://webservices.netsuite.com/wsdl/v2023_1_0/netsuite.wsdl</wsdlUrl>
                        </wsdlUrls>
                        <sourceDestDir>${project.build.sourceDirectory}</sourceDestDir>
                        <extension>true</extension>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

Similar to this the Oracle NetSuite documentation proposes Apache Axis to generate the stubs:

<profiles>
    <profile>
        <id>generate-axis</id>
        <properties>
            <jaxb2.version>2.5.0</jaxb2.version>
        </properties>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.axis2</groupId>
                    <artifactId>axis2-wsdl2code-maven-plugin</artifactId>
                    <version>1.8.2</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>wsdl2code</goal>
                            </goals>
                            <configuration>
                                <allPorts>true</allPorts>
                                <generateAllClasses>true</generateAllClasses>
                                <packageName>org.openwms.netsuite.axis2</packageName>
                                <wsdlFile>https://webservices.netsuite.com/wsdl/v2023_1_0/netsuite.wsdl</wsdlFile>
                                <databindingName>xmlbeans</databindingName>
                                <syncMode>sync</syncMode>
                                <generateServerSide>true</generateServerSide>
                                <overWrite>true</overWrite>
                                <targetResourcesFolderLocation>gen-resources</targetResourcesFolderLocation>
                                <unpackClasses>true</unpackClasses>
                                <generateServicesXml>false</generateServicesXml>
                                <generateServerSideInterface>false</generateServerSideInterface>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

All this ends up with around 1800 new classes that need to be compiled and bound into the project. The NetSuiteService class is the @WebServiceClient that can be used further on.