Bug 1707834 - Avoid libmozgtk not being linked to libxul. r=firefox-build-system-reviewers,mhentges

When the linker defaults to --as-needed, libmozgtk ends up not linked
to libxul because libxul doesn't use anything from it.
We solve the problem by adding a symbol to mozgtk and have libxul use
it.

In bug 1377445, we moved XShmQueryExtension to mozglue. While libxul
currently doesn't use the symbol (it's a workaround for system
libraries), we can move the function back to mozgtk and add a dummy
call.

Differential Revision: https://phabricator.services.mozilla.com/D113487

mozglue/build/moz.build

@@ -35,11 +35,6 @@ if CONFIG["MOZ_TSAN"]:
         "TsanOptions.cpp",
     ]
 
-if CONFIG["MOZ_WIDGET_TOOLKIT"] == "gtk":
-    SOURCES += [
-        "gtk.c",
-    ]
-
 if CONFIG["OS_TARGET"] == "WINNT":
     if CONFIG["MOZ_MEMORY"]:
         DEFFILE = "mozglue.def"

widget/gtk/mozgtk/moz.build

@@ -6,6 +6,10 @@
 
 SharedLibrary("mozgtk")
 
+SOURCES += [
+    "mozgtk.c",
+]
+
 # If LDFLAGS contains -Wl,--as-needed or if it's the default for the toolchain,
 # we need to add -Wl,--no-as-needed before the gtk libraries, otherwise the
 # linker will drop those dependencies because no symbols are used from them.

widget/gtk/mozgtk/mozgtk.c

@@ -6,8 +6,6 @@
 
 #include "mozilla/Types.h"
 
-// Only define the following workaround when using GTK3, which we detect
-// by checking if GTK3 stubs are not provided.
 #include <X11/Xlib.h>
 // Bug 1271100
 // We need to trick system Cairo into not using the XShm extension due to
@@ -16,4 +14,8 @@
 // So we define our own stub that always indicates XShm not being present.
 // mozgtk loads before libXext/libcairo and so this stub will take priority.
 // Our tree usage goes through xcb and remains unaffected by this.
-MFBT_API Bool XShmQueryExtension(Display* aDisplay) { return False; }
+//
+// This is also used to force libxul to depend on the mozgtk library. If we
+// ever can remove this workaround for system Cairo, we'll need something
+// to replace it for that purpose.
+MOZ_EXPORT Bool XShmQueryExtension(Display* aDisplay) { return False; }

widget/gtk/nsWindow.cpp

@@ -5137,6 +5137,9 @@ nsresult nsWindow::Create(nsIWidget* aParent, nsNativeWidget aNativeParent,
       // vblank.
       SetCompositorHint(GTK_WIDGET_COMPOSIDED_ENABLED);
     }
+    // Dummy call to a function in mozgtk to prevent the linker from removing
+    // the dependency with --as-needed.
+    XShmQueryExtension(mXDisplay);
   }
 #  ifdef MOZ_WAYLAND
   else if (GdkIsWaylandDisplay()) {