When checking for svSingleRef or svDoubleRef, also check for
svExternalSingleRef or svExternalDoubleRef. Otherwise the dummy reference is
treated.