Since you mention that the Pi's are accessible from the Windows host but are not on the Internet, I'm going to make an assumption that they are either on a separate VLAN or have firewall rules to prevent access from devices outside the local network.
However, Ubuntu running on WSL2 is actually on a separate network from the Windows host. Its network is running behind a NAT'd, virtual Hyper-V switch.
This is likely why Ubuntu isn't able to connect to the Pi's. If that's the case, it's pretty much the converse of this question, but with similar answers.
This should also explain the fact that name resolution is failing. The .local TLD indicates that the Pi's are using mDNS. I'm not an expert in mDNS, but I'm guessing you'd need to be able to see them on the network in order to get their addresses that way. There's no "DNS server" that Ubuntu/WSL2 could use to perform the lookup for .local.
With that said, here are some possible options (and non-options):
Run the Windows SSH client through Ubuntu
This one started off as an afterthought that I was adding at the bottom of this answer, but it's possible that it may be the most straightforward solution.
Try:
ssh.exe -X pi[i]@rpi[i].local
Since Ubuntu can run Windows commands (as long as you append the .exe, you may be able to just utilize this form to accomplish your task.
One potential caveat is that you have the -X option in there for X11 forwarding. If you really do need this to forward to Ubuntu (via WSLg), then this may not be an option. I haven't tested it in that capacity.
WSL1
Unfortunately, for your use-case, you may not be able to use WSL1 here. Normally this is the easiest method for solving this type of problem. WSL1 operates as a pseudo-kernel translation layer directly to the Windows kernel. As a result, it shares the same network as the Windows host.
However, WSL1 doesn't support mDNS, so if you want to use WSL1, you'd need to either use the IP addresses or set up some type of resolver for those Pi addresses. This should be possible (IIRC) by editing the Windows host file (c:\Windows\System32\Drivers\etc\hosts).
If you want to use this method, see the other answer I linked above for how to backup and convert to WSL1.
SSH Jumphost through Windows OpenSSH server
I find having the Windows OpenSSH server to be a nice "Swiss Army Knife" for connectivity assistance. In this case, the concept is the same as in the other answer, but the flow is reversed.
After configuring the Windows OpenSSH server as mentioned in the other answer, you would be able to access your Pi's through something like:
ssh -X pi[i]@rpi[i].local -J $(hostname).local
If your username differs for Windows, you can use:
ssh -X pi[i]@rpi[i].local -J <windows_username>@$(hostname).local
What this does is first connect to the Windows OpenSSH server using mDNS ($(hostname).local), then uses that as a jumphost to the Pi. Since you will be connected to the Windows network after the "jump", the mDNS lookup for the Pi's should work as well. I tested this the best I could with my network.
Bridge network
As mentioned in the other answer, it may be possible to use an experimental feature to create the virtual network switch in bridged mode. You no longer need a Preview version of WSL, though - As long as you've installed WSL from the Microsoft Store, it should work.
PowerShell and/or Windows SSH client through Tmux
Similar to the first possible solution and untested for this particular use-case, but in theory you can use the concept in my answer here to handle this by running PowerShell (or the Windows SSH client) inside Tmux in Ubuntu.
