Friday, October 30, 2015

The empty String madness

Introduction

When we submit a HTML form with empty input fields which are bound to non-primitive bean properties, we'd rather like to keep them null instead of being polluted with empty strings or zeroes. This is very significant as to validation constraints such as @NotNull in Bean Validation and NOT NULL in relational databases. Across years and JSF/EL versions this turned out to be troublesome as not anyone agreed on each other. I sometimes even got momentarily confused myself when it would work and when not. I can imagine that a lot of other JSF developers have the same feeling. So let's do some digging in history and list all the facts and milestones in one place for best overview, along with an useful summary table with the correct solutions.

JSF 1.0/1.1 (2004-2006)

Due to the nature of HTTP, empty input fields arrive as empty strings instead of null. The underlying servlet request.getParameter(name) call returns an empty string on empty input fields. Nothing to do against, that's just how HTTP and Servlets work. A value of null represents the complete absence of the request parameter, which is also very significant (e.g. the servlet could this way check if a certain form button is pressed or not, irrespective of its value/label which could be i18n'ed). So we can't fix this in HTTP/Servlet side and have to do it in MVC framework's side. To avoid the model being polluted with empty strings, you would in JSF 1.0/1.1 need to create a custom Converter like below which you explicitly register on the inputs tied to java.lang.String typed model value.

public class EmptyToNullStringConverter implements Converter {

    @Override
    public Object getAsObject(FacesContext facesContext, UIComponent component, String submittedValue) {
        if (submittedValue == null || submittedValue.isEmpty()) {
            if (component instanceof EditableValueHolder) {
                ((EditableValueHolder) component).setSubmittedValue(null);
            }

            return null;
        }

        return submittedValue;
    }

    @Override
    public String getAsString(FacesContext facesContext, UIComponent component, Object modelValue) {
        return (modelValue == null) ? "" : modelValue.toString();
    }

}

Which is registered in faces-config.xml as below:

<converter>
    <converter-id>emptyToNull</converter-id>
    <converter-class>com.example.EmptyToNullStringConverter</converter-class>
</converter>

And used as below:

<h:inputText value="#{bean.string1}" converter="emptyToNull" />
<h:inputText value="#{bean.string2}" converter="emptyToNull" />
<h:inputText value="#{bean.string3}" converter="emptyToNull" />

The converter-for-class was not supported on java.lang.String until JSF 1.2.

The non-primitive numbers wasn't a problem in JSF 1.x, but only in specific server/EL versions. See later.

JSF 1.2 (2006-2009)

Since JSF 1.2, the converter-for-class finally supports java.lang.String (see also spec issue 131). So you can simply register the above converter as below and it'll get automatically applied on all inputs tied to java.lang.String typed model value.

<converter>
    <converter-for-class>java.lang.String</converter-for-class>
    <converter-class>com.example.EmptyToNullStringConverter</converter-class>
</converter>
<h:inputText value="#{bean.string1}" />
<h:inputText value="#{bean.string2}" />
<h:inputText value="#{bean.string3}" />

Tomcat 6.0.16 - 7.0.x (2007-2009)

Someone reported Tomcat issue 42385 wherein EL failed to set an empty String value representing an integer into a primitive int bean property. This uncovered a long time RI bug which violated section 1.18.3 of EL 2.1 specification.

1.18.3 Coerce A to Number type N

  • If A is null or "", return 0.
  • ...

In other words, when the model type is a number, and the submitted value is an empty string or null, then EL should coerce all integer based numbers int, long, Integer, Long and BigInteger to 0 (zero) before setting the model value. The same applies to decimal based numbers float, double, Float, Double and BigDecimal, which will then be coerced to 0.0. This was not done rightly in Oracle (Sun) nor in Apache EL implementations at the date. They both just set null in the number/decimal typed model value and only Apache EL failed on primitives whereas Oracle EL properly set the default value of zero (and hence that Tomcat issue report).

Since Tomcat 6.0.16, Apache EL started to set all number/decimal typed model values with 0 and 0.0 respectively. That's okay for primitive types like int, long, float and double, but that's absolutely not okay for non-primitive types like Integer, Long, Float, Double, BigInteger and BigDecimal. They should stay null when the submitted value is empty or null. The same applies to Boolean fields which got a default value of false and Character fields which got a default value of \u0000.

So I created JSP spec issue 184 for that (EL was then still part of JSP). This coercion doesn't make sense for non-primitives. The issue got a lot of recognition and votes. After complaints from JSF users, since Tomcat 6.0.17 a new VM argument was added to disable this Apache EL behavior on non-primitive number/decimal types.

-Dorg.apache.el.parser.COERCE_TO_ZERO=false

It became the most famous Tomcat-specific setting among JSF developers. It even worked in JBoss and all other servers using Apache EL parser (WebSphere a.o). It could even be set programmatically with help of a ServletContextListener.

@WebListener
public class Config implements ServletContextListener {

    @Override
    public void contextInitialized(ServletContextEvent event) {
        System.setProperty("org.apache.el.parser.COERCE_TO_ZERO", "false");
    }

    @Override
    public void contextDestroyed(ServletContextEvent event) {
        // NOOP.
    }

}

JSF 2.x (2009-current)

To reduce the EmptyToNullStringConverter boilerplate, JSF 2.0 introduced a new context param with a rather long name which should achieve exactly the desired behavior of interpreting empty string submitted values as null.

<context-param>
    <param-name>javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL</param-name>
    <param-value>true</param-value>
</context-param>

To avoid non-primitive number/decimal typed model values being set with zeroes, on Tomcat and clones you still need the VM argument for the Apache EL parser as explained in the previous section. See also a.o. the Communication in JSF 2.0 article here.

EL 3.0 (2013-current)

And then EL 3.0 was introduced as part of Java EE 7 (which also covers JSF 2.2). With this version, the aforementioned JSP spec issue 184 was finally fixed. EL specification does no longer require to coerce non-primitive number/decimal types to zero. Apache EL parser was fixed in this regard. The -Dorg.apache.el.parser.COERCE_TO_ZERO=false is now the default behavior and the VM argument became superflous.

However, the EL guys went a bit overboard with fixing issue 184. They also treated java.lang.String the same way as a primitive! See also section 1.23.1 and 1.23.2 of EL 3.0 specification (emphasis mine):

1.23.1 To Coerce a Value X to Type Y

  • If X is null and Y is not a primitive type and also not a String, return null.
  • ...

1.23.2 Coerce A to String

  • If A is null: return “”
  • ...

They didn't seem to realize that coercion can work in two ways: when performing a "get" and when performing a "set". Coercion from null string to empty string makes definitely sense during invoking the getter (you don't want to see "null" being printed over all place in HTML output, right?). Only, it really doesn't make sense during invoking the setter (as the model would be polluted with empty strings over all place).

And suddenly, the javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL didn't have any effect anymore. Even when JSF changes the empty string submitted value to null as instructed, EL 3.0 will afterwards coerce the null string back to empty string again right before invoking the model value setter. This was first noticeable in Oracle EL (WildFly, GlassFish, etc) and only later in Apache EL (see next chapter). This was discussed in JSF spec issue 1203 and Mojarra issue 3075, and finally EL spec issue 18 was created to point out this mistake in EL 3.0.

Until they fix it, this could be workarounded with a custom ELResolver for common property type of java.lang.String like below which utilizes the new EL 3.0 introduced ELResolver#convertToType() method. The remainder of the methods is not relevant.

public class EmptyToNullStringELResolver extends ELResolver {

    @Override
    public Class<?> getCommonPropertyType(ELContext context, Object base) {
        return String.class;
    }

    @Override
    public Object convertToType(ELContext context, Object value, Class<?> targetType) {
        if (value == null && targetType == String.class) {
            context.setPropertyResolved(true);
        }

        return value;
    }

    @Override
    public Iterator<FeatureDescriptor> getFeatureDescriptors(ELContext context, Object base) {
        return null;
    }

    @Override
    public Class<?> getType(ELContext context, Object base, Object property) {
        return null;
    }

    @Override
    public Object getValue(ELContext context, Object base, Object property) {
        return null;
    }

    @Override
    public boolean isReadOnly(ELContext context, Object base, Object property) {
        return true;
    }

    @Override
    public void setValue(ELContext context, Object base, Object property, Object value) {
        // NOOP.
    }

}

Which is registered in faces-config.xml as below:

<application>
    <el-resolver>com.example.EmptyToNullStringELResolver</el-resolver>
</application>

This was finally fixed in Oracle EL 3.0.1-b05 (July 2014). It is shipped as part of a.o. GlassFish 4.1 and WildFly 8.2. So the above custom ELResolver is unnecessary on those servers. Do note that you still need to keep The Context Param With The Long Name in EL 3.0 reagardless of the fix and the custom ELResolver!

Tomcat 8.0.7 - 8.0.15 (2014)

Apache EL 3.0 worked flawlessly until someone reported Tomcat issue 56522 that it didn't comply the new EL 3.0 requirement of coercing null string to empty string, even though that new requirement didn't make sense. So since Tomcat 8.0.7, Apache EL also suffered from this EL 3.0 problem of unnecessarily coercing null string to empty string during setting the model value. However, the above EmptyToNullStringELResolver workaround in turn still failed in all Tomcat versions until 8.0.15, because it didn't take any custom ELResolver into account. See also Tomcat issue 57309. This was fixed in Tomcat 8.0.16.

If upgrading to at least Tomcat 8.0.16 in order to utilize the EmptyToNullStringELResolver is not an option, the only way to get it to work is to replace Apache EL by Oracle EL in Tomcat-targeted JSF web applications. This can be achieved by dropping the current latest release in webapp's /WEB-INF/lib (which is javax.el-3.0.1-b08.jar at the time of writing) and adding the below context parameter to web.xml to tell Mojarra to use that EL implementation instead:

<context-param>
    <param-name>com.sun.faces.expressionFactory</param-name>
    <param-value>com.sun.el.ExpressionFactoryImpl</param-value>
</context-param>

Or when you're using MyFaces:

<context-param>
    <param-name>org.apache.myfaces.EXPRESSION_FACTORY</param-name>
    <param-value>com.sun.el.ExpressionFactoryImpl</param-value>
</context-param>

Of course, this is also a good alternative to the custom EmptyToNullStringELResolver in its entirety. Also here, you still need to keep The Context Param With The Long Name.

Summary

Here's a summary table which should help you in figuring out what to do in order to keep non-primitive bean properties null when the submitted value is empty or null (so, to avoid pollution of model with empty strings or zeroes over all place).

Note: Tomcat and JBoss use Apache EL, and GlassFish and WildFly use Oracle EL. Other servers (mainly the closed source ones such as WebSphere, WebLogic, etc) are not covered as I can't tell the exact versions being affected, but generally the same rules apply depending on the EL implementation being used.

JSFTomcatJBoss ASWildFlyGlassFish
5.5.x-6.0.156.0.166.0.17+7.0.x8.0.0~68.0.7~158.0.16+9.0.x10.0.x10.1.x4.x/5.05.1+6.x/7.x8.0/8.18.2-13.x14-2122-2627+3.x4.04.1+5.x6.x7.x
1.0-1.1MCUTMC,CZMC,CZMCMC,UEMC,ERMC,ERMCMC,CZMC,CZMC,ERMCMCMCMC,ERMCMC
1.2ACUTAC,CZAC,CZACAC,UEAC,ERAC,ERACAC,CZAC,CZAC,ERACACACAC,ERACAC
2.0-2.1JFUTJF,CZJF,CZJFJF,UEJF,ERJF,ERJFJF,CZJF,CZJF,ERJFJFJFJF,ERJFJF
2.2JF,CZJFJF,UEJF,ERJF,ERJF,CZJF,ERJFJFJFJF,ERJFJF
2.3JF,ERJFJF
3.0JK*JK*JKJKJKJK
4.0JK*JKJK
  • MC: manually register EmptyToNullStringConverter over all place in <h:inputXxx converter>.
  • AC: automatically register EmptyToNullStringConverter on java.lang.String class.
  • UT: upgrade Tomcat to at least 6.0.17 as version 6.0.16 introduced the broken behavior on non-primitive number/decimal types and the VM argument was only added in 6.0.17.
  • CZ: add -Dorg.apache.el.parser.COERCE_TO_ZERO=false VM argument.
  • JF: add javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=true context param.
  • JK: add jakarta.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=true context param.
  • JK*: add jakarta.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL=true context param, and when using Mojarra, use a minimum of 3.0.4 and 4.0.1; for older versions you need a EmptyToNullStringELResolver.
  • ER: register EmptyToNullStringELResolver, or alternatively, just do UE.
  • UE: migrate/upgrade to Oracle EL implementation version 3.0.1-b05 or newer.
  • : this JSF version is not supported on this server anyway.

14 comments:

Senthil said...

Nice detailed article.
I have question, what if have converter and a custom Validator (i.e. does validate required field, length and range). First the converter converts "" to null, when I bind the validator to the inputtext it didn't validate.

BalusC said...

@Senthil: Validator won't be hit when javax.faces.VALIDATE_EMPTY_FIELDS=false. Nonetheless, just use required="true" for empty inputs and f:validateLength, f:validateDoubleRange, etc for others.

Martin G said...

Great article, perfect summary! thanks!

Ted S. said...

Thank for the great article Bauke. I just did a test with TomEE 7.0.0-M3 and it coerced empty strings on the setter to null using only:

javax.faces.INTERPRET_EMPTY_STRING_SUBMITTED_VALUES_AS_NULL
true

Vernon Singleton said...

I wonder how many reads this post has had? Good job, BalusC!

BalusC said...

@Vernon: as of today, 5662 (in 8 months). Not as much as I would hope though ;)

Anonymous said...

Great article. Can we use similar hack as EmptyToNullStringELResolver for Booleans too for JSf 2.2.

java.lang.NullPointerException
at org.apache.myfaces.view.facelets.tag.TagAttributeImpl.getBoolean(TagAttributeImpl.java:143)

Unknown said...

Thanks. Great post. We had a similar issue with Boolean in Websphere with Myfaces 2.2. We were resolve it by returning "false" for null Boolean class.

Unknown said...

Thanks. This was very useful. We had a similar issue with Boolean in Websphere with Myfaces 2.2. We were resolve it by returning "false" for null Boolean class.

Vik said...

Thanks. I hope there is more then just 5k hits as it was in July2016.

Unknown said...

It's a great post and is going to help us a lot.
Thanks U.

matsa said...

I'd love for you to add instructions for JSF 2.3 and Tomcat9, BalusC!

I recently migrated from Tomcat 8.5 to 9 and encountered this. I even created a SO question because I forgot that one already existed and didn't find it:

https://stackoverflow.com/questions/53759078/problem-with-null-versus-empty-string-when-updating-to-jsf-2-3-mojarra-on-tomc

William said...

I have a related question. When a Date field is empty on the .xhtml, the JSF HTML converter is called but passed without throwing exception. Then, the Date field custom validator is called and it hit the NullPointerException because of the use of SimpleDate java class constructor requires a non-null value and thus threw the exception instead. I am not sure why the custom validator is even called in this case. Is it because the JSF Data converter did not throw the needed exception when navigating away from the current input page to another?

Paolo said...

this is still valid with Tomcat 10.1.4 and Faces 4.0 !

the only thing I've change is the firm of the method to comply with EL 5 api

@Override
public T convertToType(ELContext context, Object value, Class targetType) {
if (value == null && targetType == String.class) {
context.setPropertyResolved(true);
return null;
}
return super.convertToType(context,value,targetType);
}