The problem is really that the paths get munged into strings in the
system codepage, losing or misinterpreting most Unicode. Python's os
module is smart enough to encode the unicode to the file system
encoding, but the bsddb module needs a little help.
Provide a new function, constfunc.get_env_var(name, default=None) to
cleanly import Unicode environment variables in Windows and use it in
place of all instances of foo = os.environ['BAR] or foo =
os.environ.get('BAR').
Os path functions are smart enough to convert unicode to the file system
encoding on their own, but Db functions aren't, so provide an _encode
function in gen.db.write.py and apply it where a path is being passed to
DBEnv.open().
Also convert paths from the UI to unicode from 'utf8' rather than
sysfilesystemencoding. The latter happens to be correct most of the time
on Linux and OSX but is wrong on Windows.
Discovered in #7530.
1) Previously, tests expected invalid BDAY with DD-MM-YYYY format
to be ignored. Now they are wrapped with datestr (see #7212),
and that is already covered by
test_birthday_invalid_format_converted_to_datestr
Corrected test expectations by removing the obsolete test.
2) Test expects that an empty BDAY record won't create any
event objects. Modified import code accordingly -- we used
to create an event object without date, regression in #7212.
This replaces commit 8cbb3ef014 with
removing the incorrect encoding of paths with the system filesystem
encoding. Python is mostly smart enough to do that itself.
Thanks to Paul Franklin and Josip Pisoj for their help with sorting this
out.
Much worse, actually. Gramps wasn't able to handle any non-ascii
characters in any preference setting from Python2.
repr() in Py2 effectively runs "encode(val, ascii, backslashreplace)"
on its argument, and there's no way to reconstruct the string.
If ICU is available, inspect the environment for $COLLATION and on
a Mac also check the AppleCollationOrder default. Pass the results
to a special ICU Locale used for sorting.
