Introducing Seren, the serialization speed enhancer

Seren (SERialization ENhancer) aims to enhance your classes so that they are much quicker to serialize.
It does so by instrumenting the classes at load-time to generate optimized writeObject / readObject methods, based on the best known practices (as seen in the JavaSpecialist Master Course).

The source code available on GitHub.
Feel free to take a look at it, any comment appreciated !


How does it work ?
Which classes should be enhanced is determined by a "filter", which is configured in the "seren.properties" configuration file (see below).
In each selected class, Seren will detect and optimize all non-static, non-transient fields. Also, final fields are not supported, because only the standart serialization system is allowed to re-set the value of final fields ; if you wish to serialize a class with final fields, please hide it from Seren.

How much faster will my objects be serialized ?
It entierely depends on the types of their fields. Numeric wrappers (Integer, Double...) and Strings are much faster (up to 3 times faster according to my – not necessarily accurate – benchmarks). Primitives are only as fast as usual.

Is there any risk / downside ?
There is absolutely no risk in testing Seren. As a Java Agent, it instruments your classes at load-time, and it totally transparent to your application. No need to modify any existing code either. Try it and measure the speed gain ; if it does not convince you, just remove the Java Agent option from the command line and you're done.

As for downsides, Seren provides a boost in serialization speed, but may generate slightly bigger serialized streams, so you sould not use it if you send big amounts of serialized data over a slow network. On the other hand, Seren works wonders for in-memory serialization (ex: EHCache make deep copies of your objects by serializing them to an in-memory byte array).

Building Seren

To compile the library and package it as a jar :

mvn clean compile package

To run the integration tests (after the library has been packaged) :

mvn verify

Also, you might want to generate the javadocs, especially if you develop a custom ClassFilter :

mvn javadoc:javadoc

Configuration

To know which classes to instrument, Seren needs you to configure a "class filter". Some of the most commonly needed are provided for your convenience (see the javadoc for a list of available filters and their configuration options), but you can very easily write your own if required.

A class filter is defined by a logical name, and an optional set of configuration properties specific to this filter.

All this configuration takes place in the "seren.properties" file, which must be placed at the root of the classpath. You can configure several filters in this file; only the one specified by the "seren.filter" property will be used.

The syntax is easy :

seren.filter=<filterId>

filter.<filterId>=com.company.filterClass
filter.<filterId>.property1=value1
filter.<filterId>.property2=value2
...

For example :

// Configure which filter will be used
seren.filter=filterByPackageList

// The selected filter's configuration
filter.filterByPackageList=net.thecodersbreakfast.seren.filter.PackageListFilter
filter.filterByPackageList.packages=com.company.pkg1, com.company.package2

// Another filter configuration - won't be used
filter.filterByPackagePattern=net.thecodersbreakfast.seren.filter.PackagePatternFilter
filter.filterByPackagePattern.pattern=^com\\.company\\.(.*)\\.model

One othe configuration option is the "seren.verbose" parameter, which can be set to true or false (default). In verbose mode, both the filter and the transformer print extra information on the standart output stream (console).

seren.verbose=true

Running Seren

To run your application with Seren, just add the following option to the command line. Also, make sure the Javassist library (javassist.jar) is available in the classpath.

-javaagent:<path/to/seren.jar>

For example :

java -cp <classpath> -javaagent:/home/olivier/seren.jar

Developing a custom class filter

Class filters must implement the net.thecodersbreakfast.seren.filter.ClassFilter interface, that defines two methods :

public void configure(Map<String,String> config) throws Exception;
public boolean acceptClass(ClassLoader classLoader, CtClass classDefinition) throws Exception;

The "setVerbose()" and "configure()" method are called after the filter is instanciated. The Map passed as a parameter contains the filter's properties defined in the configuration file ; its keys are the names of the properties related to this particular filter, minus the filter's prefix (filter.<filterId>).

The "acceptClass()" method is then called for each loaded class. It's up to you to use the provided class definition (a Javassist CtClass instance) to determine if it should be instrumented. It is recommended to used BaseClassFilter (net.thecodersbreakfast.seren.filter.BaseClassFilter) as a superclass for all filters, as it provides utility methods and performs some basic checks, such as verifying if the class is actually a class (not an enum, interface, etc.) and if it implements Serializable.

As an example, below is the code of the PackageListFilter filter :

public class PackageListFilter extends BaseClassFilter {
 
    private Set<String> packages = new HashSet<String>();
 
    @Override
    public void configure(Map<String, String> config) {
        String packageNames = config.get("packages");
        packages.addAll(Arrays.asList(packageNames.split(",\\s+")));
    }
 
    @Override
    public boolean acceptClass(ClassLoader classLoader, CtClass classDefinition) throws Exception {
        return super.acceptClass(classLoader, classDefinition) && packages.contains(classDefinition.getPackageName());
    }
}

Licence & Contact info

This library is licenced under the 3-clause BSD Licence (see the LICENCE file shipped with the source code).

IANAL, but this means (roughly) that you can freely use Seren in your personal or commercial product, in source or binary form, provided you distribute the unmodified licence file with it, make clear I am the original author, and do not use my name or the library name to promote your own products. Oh, and if it explodes in production, I don't have anything to do with it :)

For any question, please contact me here, at olivier /at/ thecodersbreakfast.net


Commentaires

1. Le mardi 10 janvier 2012, 13:56 par rrevol

This is an interesting lib. I would use it if it had some features that miss, according to me :
1- Primitives or wrapped primitives are too restrictive. Enable to deal with home-made types (add a plugin system to have transformers plugins).
2- manage arrays, collections (base classes to start : HashSet, TreeSet, ArrayList, LinkedList...), maybe maps...

2. Le mardi 10 janvier 2012, 22:16 par Deluxe

Thanks for this lib, interesting in itself and as an javaagent implementation example.
By the way, shouldn't split(",\\s+") be split(",\\s*)?

Merci pour cette bibliothèque, interessante en soi et comme exemple d'implementation d'un javaagent.
Petit détail en passant, split(",\\s+") me parait restrictif. Ne vaudrait-il pas mieux split(",\\s*") ?

Ps : Typos : standart, Runnning.

3. Le mardi 10 janvier 2012, 22:31 par Olivier Croisier

@rrevol: For a first release I did not want to over-engineer the library. But I find interesting that you ask for this feature, so I will work on it soon. In the meantime, if you have the opportunity to test it on a real project, I'd be happy to know the results.

@Deluxe: thank you for the typos and the regex, I'll correct them right away !

4. Le mardi 7 août 2012, 12:18 par uweschaefer

Nice idea, but a few comments:

1. Why don't you generate reflective code to set final fields?
2. what if writeObject is already implemented (maybe i missed something)

cu uwe

5. Le mardi 7 août 2012, 12:26 par Olivier Croisier

Hi Uwe,

1. Reflective code would be too slow, and using it to reassign final fields doesn't work on all VM versions.

2. Seren takes great care not to mess with any already-implemented custom serialization, so it checks if the writeObject is already implemented (see BaseClassFilter).

Ajouter un commentaire

Le code HTML est affiché comme du texte et les adresses web sont automatiquement transformées.