Instead of bothering to pass in -lxml and -lm so that the test and/or util
executables get built, just patch out building them when cross-compiling.

Change-Id: Ida38dcdc56675eca6ff9d549ce62b15da8e82f13