Creating an API Module Project

Use the scaleout-api-module archetype to create an API Module project.

Background

In this example, we will use the com.scaleoutsoftware.archetypes:scaleout-api-module archetype to create and customize an API module for a simple shopping cart. The API processor in an API module is responsible for persisting, analyzing, and reacting to API module invocations.

Prerequisites

Java 8 or higher.
Maven

Procedure

1. Create API module project

To create an API module Maven project, run the following command in your terminal:

mvn archetype:generate \
  -DarchetypeGroupId=com.scaleoutsoftware.archetypes \
  -DarchetypeArtifactId=scaleout-api-module \
  -DarchetypeVersion=2.0.0 \
  -DgroupId=com.mycompany.shopping \
  -DartifactId=ShoppingCart \
  -DsossObjectName=ShoppingCart \
  -DinteractiveMode=false

What These Options Do

Property	Description
`groupId`	Your organization or namespace
`artifactId`	Name of the generated project directory + artifact Id
`sossObjectNameName`	The class representing each ShoppingCart

This command generates a new folder (ShoppingCart) containing:

Java class stubs for your module classes
A sample client that performs a hashmap operation on the ShoppingCart object
A sample unit test that reads/writes to the hashmap in the ShoppingCart object
A pre-configured scaleoutPackage.json metadata file
A Maven project file (pom.xml) with preconfigured dependencies
A build pipeline for packaging your project for deployment

2. Customize state object

Customize the SOSS object used to store data for your module. A skeleton class named “ShoppingCart” was created by the archetype and must be modified to hold state for the module (in this case, a list of items in the shopping cart). Instances of this object will be stored in the ScaleOut StateServer cluster.

public class ShoppingCart {
    private List<CartItem> cartItemList;
    // For JSON serialized objects, an Id field is required to enable 
    // queries from the ActiveCaching UI. 
    private String Id;

    /* parameterless constructor needed for custom serialization */
    public ShoppingCart() { }

    public ShoppingCart(String id) {
        Id = id;
        cartItemList = new LinkedList<>();
    }

    public boolean addCartItem(CartItem item) {
        if(item != null)
            return cartItemList.add(item);
        return false;
    }

    public boolean removeCartItem(String id) {
        if(!id.isEmpty()) {
            Iterator<CartItem> it = cartItemList.iterator();
            while (it.hasNext()) {
                if (it.next().getId().compareTo(id) == 0) {
                    it.remove();
                    return true;
                }
            }
            return false;
        }
        return false;
    }

    public double getTotalPrice() {
        double runningTotal = 0.0d;
        for(CartItem item : cartItemList) {
            runningTotal+=item.getPrice()*item.getQuantity();
        }
        return runningTotal;
    }
}

Note

SOSS objects should also contain a string property named Id. While not strictly required, the Id property is used by the ScaleOut Active Caching UI to identify objects in the cluster for queries.

3. Implement the createObject() and getNewObjectPolicy() methods

Implement the createObject and getNewObjectPolicy methods in your API Processor. A skeleton class named “ShoppingCartApiProcessor” was precreated by the archetype and must be modified with concrete createObject() and getNewObjectPolicy() implementations. The createObject method is called when an API invocation is received and the associated SOSS object does not yet exist and the SossApiMethod.objNotFoundBehavior is set to Create (default). The getNewObjectPolicy method is called when an API invocation is recieved and either: 1) the object does not yet exist and the SossApiMethod.objNotFoundBehavior or 2) the object does not yet exist and a persistence provider, that is enabled for read-through, retrieves the persisted object.

public ShoppingCart createObject(String moduleName, String id) {
    return new ShoppingCart(id);
}
public NewObjectPolicy getNewObjectPolicy(String moduleName, String id, ShoppingCart object) {
    // return a NewObjectPolicy with an infinite timeout
    return new NewObjectPolicy() {
        @Override
        public Duration getExpirationDuration() {
            return Duration.ZERO;
        }

        @Override
        public ExpirationType getExpirationType() {
            return ExpirationType.Absolute;
        }
    };
}
The newObjectPolicy instance returned by the getNewObjectPolicy() method will have the following two properties set:

Expiration: A Duration indicating how long the object should be kept in the cluster before being automatically removed. By default, Duration.ofSeconds(0) – infinite timeout.

ExpirationType: An enum value indicating the expiration behavior for the object. Set to Sliding to reset the expiration every time the SossObject is accessed, or Absolute to keep it for a fixed duration. By default, ExpirationType.Sliding.

4. Implement the API module’s endpoints

The API module’s endpoints are implemented in the ShoppingCartApiProcessor class, decorated with @SossApiMethod annotations. This class contains methods for handling incoming API requests and interacting with the SOSS objects.

Endpoints accept an arbitrary argument value as a byte array and can return a byte array as a response.

@SossApiMethod(operationId = "addCartItem", lockingMode = ApiProcessorLockingMode.ExclusiveLock)
public InvokeResult addItemHandler(ApiProcessingContext<ShoppingCart> processingContext, ShoppingCart shoppingCart, byte[] payload) {
    Gson gson = new Gson();
    CartItem item = gson.fromJson(new String(payload, StandardCharsets.UTF_8), CartItem.class);
    boolean added = shoppingCart.addCartItem(item) ;
    return new InvokeResult() {
        @Override
        public byte[] getResult() {
            return new byte[] { (byte)(added ? 1 : 0)};
        }

        @Override
        public ProcessingResult getProcessingResult() {
            return ProcessingResult.DoUpdate;
        }
    };
}

@SossApiMethod(operationId = "removeCartItem", lockingMode = ApiProcessorLockingMode.ExclusiveLock)
public InvokeResult removeItemHandler(ApiProcessingContext<ShoppingCart> processingContext, ShoppingCart shoppingCart, byte[] payload) {
    String itemId = new String(payload, StandardCharsets.UTF_8);
    boolean removed = shoppingCart.removeCartItem(itemId);
    return new InvokeResult() {
        @Override
        public byte[] getResult() {
            return new byte[] { (byte)(removed ? 1 : 0)};
        }

        @Override
        public ProcessingResult getProcessingResult() {
            return ProcessingResult.DoUpdate;
        }
    };
}

@SossApiMethod(operationId = "getCartTotalPrice", lockingMode = ApiProcessorLockingMode.None)
public InvokeResult getTotalHandler(ApiProcessingContext<ShoppingCart> processingContext, ShoppingCart shoppingCart, byte[] payload) {
    // payload is expected to be null, ignore it.
    double cartTotalPrice = shoppingCart.getTotalPrice();
    return new InvokeResult() {
        @Override
        public byte[] getResult() {
            return ByteBuffer.allocate(Double.BYTES)
                    .putDouble(cartTotalPrice)
                    .array();
        }

        @Override
        public ProcessingResult getProcessingResult() {
            return ProcessingResult.DoUpdate;
        }
    };
}

Endpoints must be marked with the annotation @SossApiMethod, indicating the string name of the API method and its locking behavior.
Use lockingMode=ApiProcessorLockingMode.ExclusiveLock in the @SossApiMethod annotation if you need to ensure that only one request can modify the SOSS object at a time.
Use objNotFoundBehavior=ObjNotFoundBehavior.DoNotCreate in the @SossApiMethod annotation if you do not want the module to create objects that do not exist.
API endpoints must return an InvokeResult object. The following methods should be implemented to return concrete values from the invocation:
- getResult: The byte array to return to the caller (if any).
- getProcessingResult: Indicates what should be done with the object in the ScaleOut service after the API method has returned:
  - DoUpdate: Indicates that the object was (or may have been) modified and must be updated in the ScaleOut service.
  - NoUpdate: Indicates the object was not modified and does not need to be updated in the ScaleOut service. (If you are unsure of whether the object was modified, always return DoUpdate.)
  - Remove: Remove the associated SossObject from the ScaleOut cluster.

5. Invoke the API module’s endpoints

Clients should extend the ApiModuleClient base class to call invoke directly or instantiate a default ApiModuleClient and call invoke on that class instance to execute one of the API modules endpoints.

The ExampleClient demonstrates extending the ApiModuleClient and invoking the addItem, removeItem, and getTotalPrice endpoints.

public class ExampleClient extends ApiModuleClient {
    public ExampleClient(GridConnection connection, String moduleName) {
        super(connection, moduleName);
    }

    public boolean addCartItem(String cartId, CartItem item) throws ApiModuleException {
        Gson gson = new Gson();
        byte[] serializedCartItem = gson.toJson(item, CartItem.class).getBytes(StandardCharsets.UTF_8);
        byte[] ret = invoke(cartId, "addCartItem", serializedCartItem);
        return ret[0] == 0x01;
    }

    public boolean removeCartItem(String cartId, String itemId) throws ApiModuleException {
        Gson gson = new Gson();
        byte[] serializedId = itemId.getBytes(StandardCharsets.UTF_8);
        byte[] ret = invoke(cartId, "removeCartItem", serializedId);
        return ret[0] == 0x01;
    }

    public double getCartTotalPrice(String cartId) throws ApiModuleException {
        byte[] ret = invoke(cartId, "getCartTotalPrice", null);
        return ByteBuffer.wrap(ret).getDouble();
    }
}

To invoke an endpoint, supply the endpoints name, the target Id, and the arbitrary byte[] payload.
The invoke should handle exceptional behavior – an API module exception is thrown when:
- The invoke request times out
- The invoke handler throws an exception (the causal exception will be provided inside the API module exception)
- The invoke handler threw an exception while creating the object.
- The invoke handler did not exist.

6. Testing the API module

Developers can test their module in a local development environment. The project’s generated unit test demonstrates how to create and run the ModulePackage locally and how to create a local development ApiClient for invoking the API module’s endpoints.

The TestModule class demonstrates unit tests invoking the addItem, removeItem, and getTotalPrice endpoints.

// instantiate the module package
ModulePackage modulePackage = new ModulePackage();
// define the ApiModuleOptions
ApiModuleOptions<ShoppingCart> apiModuleOptions = new ApiModuleOptionsBuilder<ShoppingCart>(ShoppingCart.class).build();
// add the API module to the package
modulePackage.addApiModule("ShoppingCart", new ShoppingCartApiProcessor(), apiModuleOptions);
// run a local development package
modulePackage.runLocalDevelopmentEnvironment();

// instantiate the example client
String shoppingCartId = "ShoppingCart"+UUID.randomUUID().toString();
String cartItemId = "ItemId"+UUID.randomUUID().toString();
int cartItemQuantity = 15;
double cartItemPrice = 17.99;
double expectedTotal = cartItemQuantity * cartItemPrice;
ExampleClient exampleClient = new ExampleClient(GridConnection.connect(Constants.DEVELOPMENT_CONNECTION_STRING), "ShoppingCart");
CartItem newCartItem = new CartItem(cartItemId, cartItemQuantity, cartItemPrice);
// add the cart item
Assert.assertTrue(exampleClient.addCartItem(shoppingCartId, newCartItem));
// check the cart total price
Assert.assertEquals(expectedTotal, exampleClient.getCartTotalPrice(shoppingCartId), 1e-9);
// remove the cart item
Assert.assertTrue(exampleClient.removeCartItem(shoppingCartId, cartItemId));
// check the cart total price
Assert.assertEquals(0.0, exampleClient.getCartTotalPrice(shoppingCartId), 1e-9);

To invoke an endpoint, supply the endpoint’s name, the target Id, and the arbitrary byte[] payload.
The invoke should handle exceptional behavior – an API module exception is thrown when:
- The invoke request times out
- The invoke handler threws an exception (the causal exception will be provided inside the API module exception)
- The invoke handler specified objNotFoundBehavior to DoNotCreate.
- The ApiProcessor implementation threw an exception when createObject() was called.
- The invoke handler did not exist.

7. State Object Serialization

In Java, by default the SOSS state object is serialized using JSON serialization with the GSON serialization library.

To use a custom serializer, create an implementation of the CacheSerializer and CacheDeserializer. The following code snippets demonstrate using the Kryo serialization protocol for the Flight state object.

@Override
public byte[] serialize(ShoppingCart flight) throws SerializationException {
    ByteArrayOutputStream bos = new ByteArrayOutputStream();
    try (Output output = new Output(bos)) {
        kryo.writeObject(output, flight);
    }
    return bos.toByteArray();
}

@Override
public ShoppingCart deserialize(byte[] bytes) throws DeserializationException {
    ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
    try (Input input = new Input(bis)) {
        return kryo.readObject(input, ShoppingCart.class);
    }
}

Configure the ModuleOptions to use the newly defined custom serializer classes by instantiating the KryoSerializer and KryoDeserializer and setting them on the ModuleOptionsBuilder. Then, associate the ModuleOptions with the ModulePackage when adding the Flight module.

ApiModuleOptions<ShoppingCart> apiModuleOptions = new ApiModuleOptionsBuilder<ShoppingCart>(ShoppingCart.class)
        .setSerialization(new KryoSerializer(), new KryoDeserializer(), false)
        .build();