Why can I only start relay node and BP node with IP instead of public IP?

The documentation says to use Public IP but I cant. But it will sync with IP

If your Block Producing Node and Relay Nodes are on the same network, you can use their internal IPs in your topology file. When you register your relays on the network, you will need to use their Public IP so that other relays are able to connect to them.

isnt the IP of the node supposed to not being accessible from outside so that the relay only is able to talk to it and act as a proxy?

That is correct. You are registering your Relay Nodes and not your Block Producing Node.
Only your Relay Nodes should see and access your Block Producing Node. As for your Relay Nodes, everyone should be able to access them using the port for cnode (e.g: 6000).

I dont have cnode. at this point I only managed to compile and run the relay and the BP with the cardano node source. Relay’s topology points to the LAN IP of the BP and the BP’s topology points to the LAN IP of the relay

yea, that looks correct. Once you have everything setup, you want to use topologyUpdater.sh so that you’re Relay Nodes are able to connect to other Relays (we dont have peer-2-peer discovery yet).
You can read more on “topologyUpdater” here: https://cardano-community.github.io/guild-operators/#/Scripts/topologyupdater