Apologies for this dumb question... but..

How would I go about trying to find out if I can start up OpenWrt on a unsupported black box which has a LAN port and a USB port?

First of all: open up the box to see what is inside. SoC type, flash, memory size, serial console.
Then you would connect to the serial console and check the boot log, enter the boot loader, gather information. But make sure to read the wiki pages about the serial console first.

The main chip says

STi 7111-SUC

http://www.st.com/resource/en/data_brief/sti7111.pdf

Two small chips have ELPIDA printed on them.

Can't see anything like a serial console connection or pin holes where I could connect something...

Maybe if I start net console then something might pop up...

I can't work out what OS is installed on this but among the features listed in the spec it says:-

ST40-300 CPU running at 450 MHz, high
performance for Linux, Windows CE and OS21
based systems

The CPU seems to be a SH4 from the SuperH architecture, which seems to be supported by the linux kernel. I haven't seen any indication of that architecture to be supported in OpenWrt, so I think you would have to start with adding support for that architecture.
Some pages with information to get started:
https://elinux.org/Processors#SuperH
http://www.linux-sh.org/shwiki/FrontPage/

Thanks for the suggestion.

As a starting point how would I go about booting a supported version of Linux on this system before trying to build OpenWrt for it?

Well, usually enter the boot loader via serial console, move a kernel image into the devices RAM via TFTP (ethernet) or the serial line and boot it up.

I gave up on that black box as I couldn't any way to connect a serial console.

I'm now looking at another black box which does have an RS232 port. I can't see the CPU on this one as a heat sink seems to be glued to it. However I see a label on the motherboard which says AIR900HD. Is there any way to find out what the processor is?