Just some links and quotations from articles that understand this better than I.
Starting in ONTAP 8.3:
TR-4182: Ethernet Storage Design Considerations and Best Practices for Clustered Data ONTAP Configurations
“... with clustered Data ONTAP 8.3, each SVM maintains its own routing table ...”
So, we only need to consider the routing table of an SVM in ONTAP 8.3+
Prior to ONTAP 9.2, ONTAP used ip.fastpath.
IP fastpath is a mechanism that uses the network interface of an inbound request to send the response directly back to the MAC that sent the request, bypassing the routing table. By avoiding the routing table lookup, fastpath provides a quick access to data.
By default, the 'ip.fastpath' option is not configurable in Clustered Data ONTAP and is enabled by default in Clustered Data ONTAP 8.0.
Data ONTAP 8.3.1 and later allows for changing the ip.fastpath option from clustershell.
IP fastpath has been deprecated as of 9.2 and replaced by route caching*
*You need to be aware of this when upgrading from ONTAP 9.1 to ONTAP 9.2/9.3.
In ONTAP 9.2 and above, because of the change from the IP fastpath mechanism to route caching, you need to be more careful configuring your routes.
I.E. you don’t want a default route (0.0.0.0/0) to every gateway that needs to be accessible by an SVM with the same metric (not ‘route create -vserver SVM -destination 0.0.0.0/0 -gateway X.X.X.X -metric 20’ for everything.)
ONTAP 9.2 introduced a simplified and streamlined networking stack. This resulted in the elimination of two previous functionalities.
- IP Fastpath
- Routing groups
Impacts of Fastpath Elimination
IP Fastpath was originally created to bypass routing table look-ups when responding to client requests. Responses would be returned out of the same interface to the MAC address that sent the request. This resulted in quicker responses to the client, but would cause failures in cases where asymmetric routing was required.
Note: Asymmetric routing occurs when the ingress path for a client request differs from the path used to send the client a response.
In ONTAP 9.2 and later, IP Fastpath has been replaced by route caching that will accelerate routing lookups. However, environments where no return route has been configured might see no response to client requests until a proper return route is configured.
There are no known performance differences between IP Fastpath and the newer route caching functionality.
Impacts of Routing Group Elimination
Routing groups were originally used prior to ONTAP 8.3 by administrators to configure routes leverageable by SVM LIFs. In 8.3 and later, ONTAP continued to add routes to routing groups, but they were no longer manually configurable by administrators. As of ONTAP 9.2, Routing Groups have been completely removed.
Prior to ONTAP 9.2, these routing groups would allow traffic to ingress via a LIF, and only routes available via the routing-group that contained the LIF were leveraged. This meant only routes that had a gateway in the same subnet as the LIF were used.
With the removal of routing-groups in 9.2, ONTAP is able to leverage any route within the SVM's routing table. This means it is now possible to utilize a route that has a gateway in a subnet that does not match the subnet of the LIF that received the incoming TCP traffic. The result of this change is that it is much more likely to see TCP responses egressing out a port that differs from the port where the request was initially received.
This change in behavior can be problematic in environments where asymmetric routing is not permitted. Typically, this occurs due to issues where a firewall is bypassed in one direction and the corresponding connection is terminated.
The example below is fine if predominantly you want the default gateway to be 192.168.0.1. Because of asymmetric routing, traffic going to the 10.10.10.0 network, could come back via 192.168.0.1 if there is a valid route via that path, if no valid path via 192.168.0.1 or 172.16.20.1, it goes via 10.10.10.1.
cluster1::> route show -vserver SVM1
Vserver Destination Gateway Metric
---------- ------------- -------------- ------
0.0.0.0/0 10.10.10.1 40
0.0.0.0/0 172.16.20.1 30
0.0.0.0/0 192.168.0.1 20
Image: SVM with multiple ‘default gateways’