Hi Brent again,

I am trying to consolidate the possible problems, however, without seeing the whole network logical topology it's hard to say what's the real cause of this.

But I would like to propose a hypotheses that can be tested by you if it makes sense to you. 

Which protocol is in use for routing? If BGP, in order to balance incoming traffic the protocol can prioritize the traffic accordingly. Therefore it can choose a different path. To test this idea is it possible to shutdown all other ISP links and use only POST link? If request/response path is alive, NAT could not be the problem or solution.

I don't know too much Meraki but I can only answer one of your questions.

1- Is Meraki based on Linux kernel ? 
    Yes

I hope you can solve the situation asap.

Gökdağ 

On Mon, 8 Apr 2019 at 19:04, Brent Frère <Brent.Frere@abilit.eu> wrote:
Le 08/04/2019 à 17:06, Michel Kohl a écrit :
Hello Brent,

why not just put another router between the POST router and your network?
An another router means a supplementary device to install, maintain, upgrade on regular basis, patch for security reasons, and is a point of failure. I know this solution, but I would prefer just the POST to make proper technical choices (it looks that their Meraki choice is a disaster, and not only to this customer) or be able to configure the Meraki as expected, if possible.
Or connect the POST router to a dedicated network port of a linux "router" ?

This router has as many "ports" as you wish, but this doesn't provide the solution, as it is not hardware related but more routing/logic related.

If the router would manage directly the routable IP address, it would be a solution indeed, thanks the application of a secondary routing table containing the proper default route, for the traffic marked by IPTable because of its source IP address when leaving the router. We already do this with an another Internet access. But here, the firewall has not the routable IP address, so it doesn't work.

An alternative way would be to add a secondary IP address on the outside interface of the firewall (let's say 192.168.1.253/24 aside of the existing 254), asking POST to "expose" the host 192.168.1.253 instead of 192.168.1.254. By the same technique as above, it should be possible to route back the traffic to the appropriate gateway, in this case 192.168.1.100/24, using IPTable mark and alternative routing tables, indeed.

What a complexity just to correct bad hardware selection by this ISP...

For info, it would mean:

ip route add default via 192.168.1.100 table Alt_GW
This will add an alternative routing table with just and only a default gateway

ip rule add fwmark 666 table Alt_GW
This will force IP packets having mark set to 666 (mark of the Evil) to use the alternative routing table "Alt_GW"

iptable -t mangle -A OUTPUT -s 192.168.1.253 -j MARK --set-mark 666
This will mark the IP packets answering the incoming traffic arriving through the Meraki with mark 666.

ip route flush cache
This will put the new routing tables in force.

Indeed, I think this might be a workaround for some limitations of the Meraki, or the competences of the ISP technical support. Thanks for the idea.
It has also the advantage that the incoming traffic has real source IP addresses, so the firewall has access to this information for allowing it or not.

But however, I still have my questions: Is there a CLI on a Meraki, is it possible to configure a masquerade rule on incoming traffic, and so on.

For beginners, some supplementary info:

ip route list table Alt_GW
will show the content of Alt_GW table

ip rule

will show the effective rules

iptables-save
will show the effective iptables rules.

Thanks again for your suggestion.

Best regards,
Michel

On 08/04/2019 16:46, Brent Frère wrote:
Let's say the firewall outside interface is 192.168.1.254/24. The general Internet access (vDSL modem) is 192.168.1.1/24, connected to the firewall by Ethernet. It NATs all outgoing IP traffic to the Internet by replacing the source IP address by its own public IP address (masquerading) to make it routable on the World Wild Internet.

A new secondary Internet access is added. It is implemented by a Meraki, operated by POST. I have NO ACCESS to its configuration, not even the pppoe credentials, so no ways to circumvent it.

The Meraki LAN interface is, let's say, 192.168.1.100/24. It has a fixed public IPv4 WAN address. All incoming traffic (limited to IPv4, with a limited MTU and only ICMP, UDP or TCP because of POST limitations) is forwarded to the firewall outside IP address, 192.168.1.254/24, by POST configuration.

This way, the general traffic works as usually and previously: through the general Internet access, the default gateway, the vDSL modem. There is so in the firewall a "default route" with 192.168.1.1 as default gateway.
Some selected traffic can be routed to the POST Internet access, depending on its destination IP address or because of flagged traffic (iproute with marks and multiples routing tables to select the appropriate gateway).
As example, all the traffic to youporn (216.18.168.116) could be routed through the POST Internet access by adding a static route on the firewall:

# route add -host 216.18.168.116 gw 192.168.1.100

This traffic takes so the POST Internet access to reach its destination, and is NATed by the Meraki (source IP address NAT), so the return traffic reaches back the POST Internet access, and is reverse-NATed back to the firewall...

The problem is to try to use the POST Internet access for INCOMING traffic.

If a connection attempt comes from the Internet (let's say 1.1.1.1) through this POST Internet access, the IP packet is forwarded to the firewall ("exposed host") by the Meraki, because of POST configuration, and it is what is expected. But as this packet has still 1.1.1.1 as source IP address, the answer to this request is routed back to the Internet by the general Internet access, so will be NATed (masqueraded) from a different public IP address.
The host 1.1.1.1 will then receive answer to it's connection attempt from a completely different IP address, and so will not link it to the request, so it won't work.

The solution would be to masquerade the source IP address of the incoming traffic crossing the Meraki, so that the firewall would see it as coming from the Meraki internal IP address, in this example 192.168.1.100/24.
The firewall will then answer the request apparently coming from 192.168.1.100, and the Meraki should reverse-NAT it back to it's actual destination, in this example 1.1.1.1. The connection will establish.

For sure, I know that in this case, the firewall will not be able to filter incoming traffic based on it's source IP address, but it's just about making it possible, not (yet) clever or appropriate.

So I ask POST technical service to set-up this internal NAT, and they answer that it's NOT SUPPORTED by the Meraki. I can't believe Cisco has became so bad that they are not able to do this simple masquerading, especially as I suspect they use Linux as underlying technology...

So I looked at Meraki documentation, but I didn't found anything else than a very basic web-based configuration interface manual. That's why I ask if:
  • Somebody knows Meraki
  • Is Meraki based on Linux kernel ?
  • Is there a way to access the Meraki CLI ?
  • Is there a way to configure this very basic traffic masquerading rule ?
So that I could find a solution (at least limited so far) and learn POST help-desk again something they didn't know...

Note: changing the default route IS NOT an acceptable solution, as changing the default Internet access would have lots of other consequences, due to historical reasons.
The incoming traffic through the POST Internet access comes potentially from any public IPv4 routable address. It should be answered by the two Internet accesses.

Actually, the real situation is more complex: the firewall is exposed to incoming traffic from already three Internet accesses, provided by various ISPs and technologies. And it works, as I have control over those Internet accesses.
The problem occurs just because of this fourth "Internet access" by POST, which is out of my control and apparently unable to provide this basic feature: masquerading the incoming traffic.

By the way, I'm not even sure this "Internet access service" can be qualified of "Internet access" as anybody knows (or should know) that what is common on the Internet is... the IP protocol. However, this "Internet access" service is strictly limited to IPv4, and to only the TCP, UDP and ICMP protocols. This is so a very limited Internet access, somehow as you might have received from some hotels in old times: a "Internet access" that was limited to web browsing, and blocked any "unknown" service, such as SSH, telnet, IMAP, and so on. This also happened in the very old days (about 15 years ago) when some ISPs blocked SIP traffic or added intentionally jitter to the VoIP audio streams...

Those limitations have been ruled as illegal already, but here in Luxembourg, there is a delinquent company called POST that still apply such restrictions, apparently...

Practically, those restrictions prevents customers to access to:
  • IPSec VPNs, as some implementations requires ESP and AH protocols (not TCP or UDP)
  • GRE
  • IPinIP
  • PIM, IGMP (for multicast)
  • various routing protocols
  • IPv6
  • RDP
  • and so on...
which are part of the Internet protocols, even if unknown by POST commercial management, apparently.

I notice that some other ISPs (LOL, MixVoIP, ...) do provide genuine Internet accesses, with full 1500 MTU and all the services above IP, as expected...

Any help would be appreciated.

Le 08/04/2019 à 12:17, Gökdağ Göktepe a écrit :
I am trying to figure out your problem but French is a bit complicated for me. As an instance I think it has sth to do with administrative distance . But I don’t know if you have static routes defined for your secondary internet access and how

Gökdağ

On 8 Apr 2019, at 11:56, Brent Frère <Brent.Frere@abilit.eu> wrote:

Not yet.

Le 08/04/2019 à 11:17, Gökdağ Göktepe a écrit :
Hi Brent did you find any solution?

Gökdağ

_______________________________________________
Lilux-info mailing list
Lilux-info@lilux.lu
https://www.lilux.lu/mailman/listinfo/lilux-info


_______________________________________________
Lilux-info mailing list
Lilux-info@lilux.lu
https://www.lilux.lu/mailman/listinfo/lilux-info


_______________________________________________
Lilux-info mailing list
Lilux-info@lilux.lu
https://www.lilux.lu/mailman/listinfo/lilux-info