[SOLVED] pfsense backup route

[birne]

Hello.
i do have a strange problem.
I have a primary and a secondary pfsense in my network. the primary is working as it should and all clients can also connect to the internet,
but if i cut shut the port for the primary firewall the router injects the route to the secondary firewall into eigrp and traffic should flow to the secondary and it does for some of the devices on the lan.
all my network switches can still figure out the newly distributesd route (testet with traceroute),
but my clients/server/pc can not connect to the internet.
setup is as follows

---------- wan-ip: 10.135.0.54/24-PFSENSE1-lan-ip: 10.10.1.0/30 - eth1 on core1
WAN -------------------------------------------------------- ---------------------------------- Core1 lan: 10.10.0.0/24
---------- wan-ip: 10.135.0.60/24-PFSENSE2-lan-ip: 10.10.1.4/30 - eth2 on core1

LAN behind core 10.10.0.0/24 --switch10.10.0.2/24 -- client10.10.0.10/24

both pfsense can ping into the network and all devices on the network can ping the pfsense.
both pfsense are setup with the same routing exept the gateway for the lan is different.
core has a default static route to 10.10.1.1(pfsense1) and distributes that route via eigrp to all the switches in the network.
core is setup to track reachability to pfsense1 and inject the route to 10.10.1.5 (pfsense2) incase it is unreachable.
i checked that the new route is in the routing table and it is. A traceroute from any switch confirms traffic flows through pfsense2.
but all my clients on the 10.10.0.0 network can not ping anything on the internet e.g. 8.8.8.8
this does not make sense in my head. why can the switches with an address on the same subnet as the clients ping google but the clients cant.
the clients do have a default gateway set to one of the switches(hsrp) and not the pfsense.
pfsense has HA setup and does config sync with all but static-routes


the setup is simplified for easier explanation.

please ask away if you have questions about the setup
or give advice to what i can try to figure this out.

Thank you in advance for your help.

 

[nigelivey]

Are you using CARP?

 

[birne]

no, and i dont think it is possible in my case since the primary and secondary have different subnets on lan side.
on wan side i also have no carp.

the strange part for me is how the switch can connect to internet but pc cant.

 

[bill001g]

It has been a long time since I messed with stuff like this but it sounds like you have a async data path where the routing protocol and the HSRP are choosing different paths for the traffic depending on the direction. Sometimes that makes things not work.

Be very careful when you test be sure to specify the correct ip on each switch. Since cisco stopped support for HSRP on newer switches I have actually forgotten how it works in detail I know there was some strangeness when it came to ping. Maybe try to move to VRRP instead unless you have extremely old equipment that does not support it.

 

[birne]

all switches and my core are L3 catalyst 3750 with ip-routing on core and dist, not access
hsrp doesnt provide any path for the traffic but only provides a gw for clients on the different vlans.