We recently upgraded Mapserver to the current version for a project because we wanted the WFS id field to be populated in geojson which required a recent version of (OGR >=2.3) which was included in mapserver build 7.2.1

All went well and we modified the mapserver geojson output format to specify the field we wanted to use (originally named id in our case) but adding in the line: FORMATOPTION "LCO:ID_FIELD=id"

OUTPUTFORMAT
   NAME "geojson"
   DRIVER "OGR/GEOJSON"
   MIMETYPE "application/json; subtype=geojson"
   FORMATOPTION "STORAGE=stream"
   FORMATOPTION "FORM=SIMPLE"
   FORMATOPTION "LCO:COORDINATE_PRECISION=0"
   FORMATOPTION "LCO:ID_FIELD=id"
END

The field “id” needed to be included in the relevant map layer in the gml_include_items e.g.:

"gml_include_items"     "name,recsubtype,recsubtypecode,rectype,ref,uid,id"

After those changes – all seemed to be going well and the id was appearing in our WFS return

However when we looked at some of the other attributes we noticed scrambled characters (these had been fine with the previous version of mapserver). e.g. rectype in screenshot below

Our data was coming from SQL server using the msplugin_mssql2008 plugin– and if we set debug in the mapserver layer and looked at the log we saw:

msConvertWideStringToUTF8(): General error message. Unable to convert string in encoding 'UCS-2LE' to UTF8 An invalid multibyte sequence has been encountered in the input

So it looked as it something was going wrong in the unicode translation.  The workaround was refreshingly simple – we just had to alter the view that the data was being pulled from to force the conversion e.g. change:

‘Site’

to:

CONVERT(nvarchar(1),N’Site’)

then all was well again.