You would be able change it in the /etc/gai.conf file.
From man gai.conf:
DESCRIPTION
A call to getaddrinfo(3) might return multiple answers. According to
RFC 3484 these answers must be sorted so that the answer with the high‐
est success rate is first in the list. The RFC provides an algorithm
for the sorting. The static rules are not always adequate, though.
For this reason, the RFC also requires that system administrators
should have the possibility to dynamically change the sorting. For the
glibc implementation, this can be achieved with the /etc/gai.conf file.
Each line in the configuration file consists of a keyword and its pa‐
rameters. White spaces in any place are ignored. Lines starting with
'#' are comments and are ignored.
There is a section in that file of commented lines that you can change. There is one line in particular that states the following:
# For sites which prefer IPv4 connections change the last line to
#
#precedence ::ffff:0:0/96 100
Which should mean that if you want IPv4 preferred over IPv6 you would basically just have to uncomment the lines above and change the last line to what the comment is stating.