Discussion:
How to downgrade or patch freetype-2.7 ?
(too old to reply)
Volth
2017-04-19 11:25:54 UTC
Permalink
Hi

Freetype changed defaults in 2.7, the truetype interpreter became v40
instead of v35.
The new defaults are incomatible in a weird way with some old fonts
(notably PragmataPro).

There is upstream-recommended way to "globally" set old interpreted
version via environment variable FREETYPE_PROPERTIES, which is not a
solution obviously: many processes do not inherit parent environment
(sudo, browser renderers run in sandbox, ...) .

So fretype is to be patched and recompiled. This is simple on say
Ubuntu, make a new .deb, install it.

But on NixOS this - flipping one bit in libfreetype.so - means either
to recompile all NixOS (too many packages depends on freetype, not
only GUI apps, also devtools) or to live with corrupt nixstore.

Is there a more elegant solution?
Thomas Tuegel
2017-04-19 15:06:03 UTC
Permalink
Post by Volth
Freetype changed defaults in 2.7, the truetype interpreter became v40
instead of v35.
The new defaults are incomatible in a weird way with some old fonts
(notably PragmataPro).
But on NixOS this - flipping one bit in libfreetype.so - means either
to recompile all NixOS (too many packages depends on freetype, not
only GUI apps, also devtools) or to live with corrupt nixstore.
Is there a more elegant solution?
There is a more elegant work-around, and possibly an actual solution.

First, the workaround: add a modified FreeType library to
LD_LIBRARY_PATH. This environment variable should be passed through to
every program on NixOS because that is how we handle the impurity of
OpenGL drivers.

Second, I can probably fix the way PragmataPro renders by applying
selective Fontconfig rules. Could you please tell me, what settings are
you using in `fonts.fontconfig'? If you have changed any
Fontconfig settings like DPI or hinting through your desktop
environment, please share that also. Unfortunately, it is not possible
to simply change the interpreter from v40 to v35 through Fontconfig
rules. However, changing the hinting level for that font only can
essentially achieve the same thing.

Regards,
Tom
Volth
2017-04-19 16:12:48 UTC
Permalink
Oh, LD_LIBRARY_PATH advice is very helpful. Some minor issues left
(including one of how how to concatenate my LD_LIBRARY_PATH and
OpenGL's), but it solved the mass-rebuild problem. Now I could patch
the library or even use the good old 2.6.

Fontconfig does not help here. The problem is with all fonts with
manual hinting, also with all old windows fonts. Those fonts are from
pre-anti-aliasing ages.
Full hinting enabled + anti-aliasing disabled is the only suitable
mode for them.
Post by Thomas Tuegel
Post by Volth
Freetype changed defaults in 2.7, the truetype interpreter became v40
instead of v35.
The new defaults are incomatible in a weird way with some old fonts
(notably PragmataPro).
But on NixOS this - flipping one bit in libfreetype.so - means either
to recompile all NixOS (too many packages depends on freetype, not
only GUI apps, also devtools) or to live with corrupt nixstore.
Is there a more elegant solution?
There is a more elegant work-around, and possibly an actual solution.
First, the workaround: add a modified FreeType library to
LD_LIBRARY_PATH. This environment variable should be passed through to
every program on NixOS because that is how we handle the impurity of
OpenGL drivers.
Second, I can probably fix the way PragmataPro renders by applying
selective Fontconfig rules. Could you please tell me, what settings are
you using in `fonts.fontconfig'? If you have changed any
Fontconfig settings like DPI or hinting through your desktop
environment, please share that also. Unfortunately, it is not possible
to simply change the interpreter from v40 to v35 through Fontconfig
rules. However, changing the hinting level for that font only can
essentially achieve the same thing.
Regards,
Tom
Thomas Tuegel
2017-04-19 21:47:07 UTC
Permalink
Post by Volth
Fontconfig does not help here. The problem is with all fonts with
manual hinting, also with all old windows fonts. Those fonts are from
pre-anti-aliasing ages.
Full hinting enabled + anti-aliasing disabled is the only suitable
mode for them.
We can use Fontconfig to set per-font hinting and anti-aliasing
settings! :-)

We can't toggle the interpreter version directly through Fontconfig, but
at `full' hinting, the v40 interpreter should give the same results as
v38, especially with anti-aliasing disabled. I can't speak reliably
about the v35 interpreter because NixOS has always used v38 in the past.

I assume that "old windows fonts" means corefonts? Are you having
trouble with any other fonts besides PragmataPro? If you give me a list,
I can put together some samples to see if we can't find something better
for everyone.

Regards,
Tom
Volth
2017-04-20 05:35:29 UTC
Permalink
Better to show: http://imgur.com/a/a5XpN

The first image (http://imgur.com/zOyxMh7) is with the default
NixOS-17.09 (freetype-2.7.1) settings.
Approximately the same is on 17.03 with its defaults.
Fonts are blurry, width of cells is wrongly calculated.

This could be fixed with fonconfig, and I did fix it with fontconfig on 17.03:
-------
<match target="font">
<test name="family"> <string>PragmataPro</string>
</test>
<edit name="antialias" mode="assign"> <bool>false</bool>
</edit>
<edit name="hintstyle" mode="assign"> <const>hintfull</const>
</edit>
<edit name="autohint" mode="assign"> <bool>false</bool>
</edit>
</match>
-------
The same fontconfig settings applied to:

PragmataPro
PragmataPro Mono <- that font family

Liberation Sans
Liberation Serif
Liberation Mono <- version 1.07, not NixOS's default 2.0x

Liberastika <- fork of liberation-1.07 with some fixes

Ubuntu
Ubuntu Condensed
Ubuntu Mono <- these are not so bad with the defaults, it is just a
matter of taste to have them antialiased, especially "Ubuntu
Condensed"

Arial
Times New Roman
Gulim
Meiryo
Microsoft YaHei <- microsoft fonts, copied from windows machine,
antialiasing can make a blotch out of a Chinese hieroglyph.

(As you see, almost all these fonts are either commercial or license
violation, so no wonder gnu's freetype does not support them well)

The second (http://imgur.com/MDiydzS) is what we get with these
fontconfig rules on NixOS-17.09 (freetype-2.7.1) with default v40.
Antialias is off, but hinting is not in action.

The third (http://imgur.com/sKs8ZrB) is v35, explicitly set on
NixOS-17.09 (freetype-2.7.1) or default on NixOS-17.03
(freetype-2.6.5).
Perfect.

PS. v38 works similar to v40. It easy to turn in off by
"config.fonts.fontconfig.ultimate.enable = false;".
What I was looking for is a similar easy way to turn off v40 appeared
in NixOS-unstable.

So I ended up creating a module:

```
{ config, pkgs, lib, ... }:

let
my-freetype = pkgs.callPackage ./my-freetype { };
my-freetype-32 = pkgs.pkgsi686Linux.callPackage ./my-freetype { };
my-pragmatapro = pkgs.callPackage ./pragmatapro { };
my-win2008fonts = pkgs.callPackage ./win2008fonts { };
my-win2008fonts-cjk = pkgs.callPackage ./win2008fonts-cjk { };
in {
config = {
fonts = {
enableDefaultFonts = false;

fonts = [
pkgs.xorg.fontbh100dpi
pkgs.xorg.fontmiscmisc
pkgs.xorg.fontcursormisc
pkgs.ubuntu_font_family
pkgs.liberation_ttf
pkgs.liberastika
my-pragmatapro
my-win2008fonts
my-win2008fonts-cjk
];

fontconfig = {
hinting.autohint = false;

defaultFonts.serif = [ "Liberation Serif" "Times New Roman" ];
defaultFonts.sansSerif = [ "Liberastika" "Liberation Sans"
"Arial" "Ubuntu" ];
defaultFonts.monospace = [ "PragmataPro Mono" ];

confPackages = [ (import ./font-conf { inherit (pkgs)
runCommand fontconfig; }) ]; # it adds fontconfig xml

useEmbeddedBitmaps = true; # 17.09+
penultimate.enable = false; # 17.09+
ultimate.enable = false;
};
};

environment.sessionVariables.LD_LIBRARY_PATH = [
"${my-freetype}/lib" "${my-freetype-32}/lib" ];

nixpkgs.overlays = [
(self: super: {
# google-chrome, torbrowser (..., gorilla, dropbox,
cytrix-receiver, spideroak, ...) add freetype to LD_LIBRARY_PATH in makeWrapper
torbrowser = super.torbrowser.override {
freetype = my-freetype;
};
google-chrome = super.google-chrome.override {
freetype = my-freetype;
};
liberation_ttf = self.liberation_ttf_v1_binary;
})
];

};
}

```
Post by Thomas Tuegel
Post by Volth
Fontconfig does not help here. The problem is with all fonts with
manual hinting, also with all old windows fonts. Those fonts are from
pre-anti-aliasing ages.
Full hinting enabled + anti-aliasing disabled is the only suitable
mode for them.
We can use Fontconfig to set per-font hinting and anti-aliasing
settings! :-)
We can't toggle the interpreter version directly through Fontconfig, but
at `full' hinting, the v40 interpreter should give the same results as
v38, especially with anti-aliasing disabled. I can't speak reliably
about the v35 interpreter because NixOS has always used v38 in the past.
I assume that "old windows fonts" means corefonts? Are you having
trouble with any other fonts besides PragmataPro? If you give me a list,
I can put together some samples to see if we can't find something better
for everyone.
Regards,
Tom
Thomas Tuegel
2017-04-20 14:59:22 UTC
Permalink
Post by Volth
The second (http://imgur.com/MDiydzS) is what we get with these
fontconfig rules on NixOS-17.09 (freetype-2.7.1) with default v40.
Antialias is off, but hinting is not in action.
Did you intentionally turn off hinting in that sample, or is it just not
working? Even with v40, it should be possible to achieve good results
with full hinting and no antialiasing.
Post by Volth
PS. v38 works similar to v40. It easy to turn in off by
"config.fonts.fontconfig.ultimate.enable = false;".
What I was looking for is a similar easy way to turn off v40 appeared
in NixOS-unstable.
That setting doesn't actually turn off v38! It turns off the
fontconfig-ultimate settings, but FreeType 2.6 is still patched to use
v38 instead of v35. So, if you could get acceptable rendering with
FreeType 2.6 on NixOS (which used v38), we should also be able to fix it
with FreeType 2.7 (v40) too.
Volth
2017-04-20 15:28:03 UTC
Permalink
Post by Thomas Tuegel
Post by Volth
The second (http://imgur.com/MDiydzS) is what we get with these
fontconfig rules on NixOS-17.09 (freetype-2.7.1) with default v40.
Antialias is off, but hinting is not in action.
Did you intentionally turn off hinting in that sample, or is it just not
working? Even with v40, it should be possible to achieve good results
with full hinting and no antialiasing.
Hinting is on, the difference between the 2nd and the 3rd images is
only 35 vs. 40 in the environment variable.
If you want to reproduce, you may disable antialiasing and enable full
hinting for pkgs.ubuntu_font_family or pkgs.liberation_ttf_v1_binary
which are in nixpkgs.
Post by Thomas Tuegel
Post by Volth
PS. v38 works similar to v40. It easy to turn in off by
"config.fonts.fontconfig.ultimate.enable = false;".
What I was looking for is a similar easy way to turn off v40 appeared
in NixOS-unstable.
That setting doesn't actually turn off v38! It turns off the
fontconfig-ultimate settings, but FreeType 2.6 is still patched to use
v38 instead of v35. So, if you could get acceptable rendering with
FreeType 2.6 on NixOS (which used v38), we should also be able to fix it
with FreeType 2.7 (v40) too.
There was a visually similar issue in 17.03 with liberation_ttf_2.xx
[1] which forced me to downgrade to 1.07. In 17.09 even
liberation_ttf_1.07.xx became broken.

There are clearly two diverging ways: the mainstream with
antialiasing, subpixel rendering, gray-on-white texts, ... and the
minority people who want to stay with contrast fonts rendered using
only two colors (foreground and background). In order to stay the
ground both the fonts and freetype are to be downgraded to their
previous versions, so it won't be so long to lose the battle. Anyway,
as dpi grows and pixels became invisible, it won't be the matter at
all.

1. https://github.com/NixOS/nixpkgs/issues/21806
Volth
2017-06-24 00:52:01 UTC
Permalink
There are still at least one place where
"environment.sessionVariables.LD_LIBRARY_PATH = ..." does not replace
freetype with a custom-built version: in cgi-scripts run under
lighttpd service:

RRD rendered with stock freetype-2.7.1 Loading Image...
RRD rendered with freetype-2.7.1 compiled without subpixel rendering
(#define TT_CONFIG_OPTION_SUBPIXEL_HINTING 0):
Loading Image...
Post by Volth
Post by Thomas Tuegel
Post by Volth
The second (http://imgur.com/MDiydzS) is what we get with these
fontconfig rules on NixOS-17.09 (freetype-2.7.1) with default v40.
Antialias is off, but hinting is not in action.
Did you intentionally turn off hinting in that sample, or is it just not
working? Even with v40, it should be possible to achieve good results
with full hinting and no antialiasing.
Hinting is on, the difference between the 2nd and the 3rd images is
only 35 vs. 40 in the environment variable.
If you want to reproduce, you may disable antialiasing and enable full
hinting for pkgs.ubuntu_font_family or pkgs.liberation_ttf_v1_binary
which are in nixpkgs.
Post by Thomas Tuegel
Post by Volth
PS. v38 works similar to v40. It easy to turn in off by
"config.fonts.fontconfig.ultimate.enable = false;".
What I was looking for is a similar easy way to turn off v40 appeared
in NixOS-unstable.
That setting doesn't actually turn off v38! It turns off the
fontconfig-ultimate settings, but FreeType 2.6 is still patched to use
v38 instead of v35. So, if you could get acceptable rendering with
FreeType 2.6 on NixOS (which used v38), we should also be able to fix it
with FreeType 2.7 (v40) too.
There was a visually similar issue in 17.03 with liberation_ttf_2.xx
[1] which forced me to downgrade to 1.07. In 17.09 even
liberation_ttf_1.07.xx became broken.
There are clearly two diverging ways: the mainstream with
antialiasing, subpixel rendering, gray-on-white texts, ... and the
minority people who want to stay with contrast fonts rendered using
only two colors (foreground and background). In order to stay the
ground both the fonts and freetype are to be downgraded to their
previous versions, so it won't be so long to lose the battle. Anyway,
as dpi grows and pixels became invisible, it won't be the matter at
all.
1. https://github.com/NixOS/nixpkgs/issues/21806
Linus Heckemann
2017-06-24 10:10:50 UTC
Permalink
Post by Volth
There are still at least one place where
"environment.sessionVariables.LD_LIBRARY_PATH = ..." does not replace
freetype with a custom-built version: in cgi-scripts run under
RRD rendered with stock freetype-2.7.1 http://i.imgur.com/HVZ5iPa.png
RRD rendered with freetype-2.7.1 compiled without subpixel rendering
http://i.imgur.com/Qj4W0Lx.png
That is pretty terrible! Depending on what lighttpd does with the
environment of the scripts, one or more of the following might help (in
order of how nice I think they are as solutions):

0. Link against a working freetype in the first place
1. Set LD_LIBRARY_PATH in environment.variables instead of
environment.sessionVariables so that it affects not only interactive
sessions
2. Set systemd.services.lighttpd.environment.LD_LIBRARY_PATH
3. Use mod_setenv to set LD_LIBRARY_PATH for the CGI scripts.
4. (as an unfortunate last resort) wrap the CGI scripts.


Linus

Vladimír Čunát
2017-04-19 21:41:33 UTC
Permalink
Just my two cents.
Post by Volth
There is upstream-recommended way to "globally" set old interpreted
version via environment variable FREETYPE_PROPERTIES, which is not a
solution obviously: many processes do not inherit parent environment
(sudo, browser renderers run in sandbox, ...) .
Oh, LD_LIBRARY_PATH advice is very helpful.
I can't see how setting $LD_LIBRARY_PATH is easier than setting
$FREETYPE_PROPERTIES. I believe the latter is safer, as it should only
affect freetype.

If you want details on (some) potential dangers of $LD_LIBRARY_PATH on
nontrivial libs, you can study https://github.com/NixOS/nixpkgs/issues/16779

--Vladimir
Volth
2017-04-20 05:41:30 UTC
Permalink
Unfortunately, $FREETYPE_PROPERTIES is censored by sudo and chrome
sandbox (and the bug is WONTFIX in both freetype and chrome)
Post by Vladimír Čunát
Just my two cents.
Post by Volth
There is upstream-recommended way to "globally" set old interpreted
version via environment variable FREETYPE_PROPERTIES, which is not a
solution obviously: many processes do not inherit parent environment
(sudo, browser renderers run in sandbox, ...) .
Oh, LD_LIBRARY_PATH advice is very helpful.
I can't see how setting $LD_LIBRARY_PATH is easier than setting
$FREETYPE_PROPERTIES. I believe the latter is safer, as it should only
affect freetype.
If you want details on (some) potential dangers of $LD_LIBRARY_PATH on
nontrivial libs, you can study
https://github.com/NixOS/nixpkgs/issues/16779
--Vladimir
Loading...