introduction¶
Bird is a routing daemon. It can do different routing protocols, at the moment, we are using it to do BGP, for both ipv4 and ipv6.
Bird runs as a daemon as the bird user, it has one daemon and set of configurations for ipv4, and one for ipv6 (bird6).
There is a CLI for interacting with both bird daemons, for the ipv4 one, you can get into it by typing ‘birdc’ as root, for the ipv6 one you get there by typing ‘birdc6’. These have rudimentary readline support, and you can hit to complete, and ? for help in the middle of a command (to see what subcommand options are available). Additionally, these CLIs can take stdin and spit something out, for example, you can do the following in the shell:
wren# birdc show route |grep '!'(! can mean that bird tries to add a route to the kernel, while it’s already there (like, set manually with ip r add, instead of by bird)
The bird configuration is located in /etc/bird, and contains a main ‘bird.conf’ (and bird6.conf) config file. As of this writing, the configuration files have been split into a ‘peers.conf’ and a ‘filters.conf’, these are included by the bird.conf. The primary bird.conf file has the generic configuration, the peers.conf has the peer bgp session configurations, and then filters.conf has the different filters that are referenced by the different peers.
a note about bgp¶
We have coordinated with upstreams to establish BGP sessions. We did that by asking them to establish a bgp session with us (and we arranged a password with them) and then we configure bird to connect, over TCP (port 179) to their router. A BGP session is up when the peers are connected to each other, are communicating properly and have authenticated.
We then ask our peers to accept our announcements of netblocks that we have allocated to us. In some cases, we needed to prove that we were the authorized organization who is authoritative for that netblock. If they are good, they will not accept any announcements from us that are for any network that is not one of our pre-arranged, and authorized netblocks.
Finally, we determine how much of a BGP feed we get from the peers that we have an established session with. We have chosen the full firehose. This BGP feed contains all the routes on the internet (in the case of cogent, we also get a ipv6 feed). The BGP feed contains all the routes on the internet that that peer knows and their current state. That feed can contain a number of things that we do not want, such as our own announced netblocks (we are telling them, we don’t want them to tell us), bogons, default routes, etc. So we need to filter those out so we don’t accept them.
We have multiple BGP sessions configured, each receives its own copy of the remote’s BGP tables, and those are merged in our system to then determine the best path for various networks that we might like to communicate with. These routes are inserted into the kernel on the router. That means that the system no longer has a simple ‘ip route ls’ result of maybe 5 lines, it now has 400k lines, in otherwords the entire internet.
Until the BGP session is up, and our peer is receiving our announcements, our IPs are not available on the internet, at all. This is important to know! It means that when the router reboots, all of the netblocks that we are announcing will not be available until bird comes up, it connects to the other peer(s) and it begins to tell them that we are the place to go for that netblock. If the router comes up, and there is a configuration error, or it doesn’t start for some reason or another, those ip blocks will not be available! Until then, the only way to get at the router is through one of the netblocks that we are not announcing.
some useful commands¶
What follows are some useful bird commands that I’ve found useful in setting things up and debugging. Its by no means exhaustive, but covers the most common situations.
To initiate these commands you need to run the ‘birdc’, or ‘bird6’ command discussed above while on the router and you will immediately get the bird prompt:
root@wren:/home/micah# birdc
BIRD 1.4.4 ready.
bird>
You are now inside the matrix. You can leave at any time by hitting control-D, or typing exit, quit.
show the BGP peers and their state:¶
bird> show protocols
name proto table state since info
cogent BGP master up 2014-10-08 Established
cogent_blackhole BGP master up 2014-10-08 Established
swiftco BGP master up 2014-10-08 Established
bgp_spamd BGP t_spamd up 2014-10-09 Established
kernel1 Kernel master up 2014-10-07
device1 Device master up 2014-10-07
static_bgp Static master up 2014-10-07
bird>
The above is the abbreviated summary of the different configured protocols, there are four BGP sessions that are all up and established.
bird> show protocols all
name proto table state since info
cogent BGP master up 2014-10-08 Established
Description: Cogent
Preference: 500
Input filter: bgp_in_cogent
Output filter: bgp_out
Routes: 504759 imported, 3 exported, 504759 preferred
Route change stats: received rejected filtered ignored accepted
Import updates: 2618319 0 3 1052269 1566047
Import withdraws: 117315 0 --- 7 117311
Export updates: 2130661 1566184 564474 --- 3
Export withdraws: 203425 --- --- --- 0
BGP state: Established
Neighbor address: 38.104.127.1
Neighbor AS: 174
Neighbor ID: 66.28.1.143
Neighbor caps: refresh AS4
Session: external AS4
Source address: 38.104.127.2
Hold timer: 141/180
Keepalive timer: 25/60
...
The above is an abbreviated list of the details of all the protocols. I am only showing you one peer, and cutting the rest for length. Its showing that this BGP session is Established, how long its been up for, which filters are being applied, the neighbor’s IP address, ASN, and the various number of routes pulled in, and announced out. This is the state you want a bgp session to be in. If it is not Established with route changes, then it is not up.
show what BGP peers are receiving announcements for a specific netblock¶
unsure how to do this in bird
show what routes we are exporting to a peer¶
bird> show route for 198.252.153.0/24
198.252.153.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
bird> show route all export cogent
204.13.164.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
Type: static unicast univ
BGP.origin: IGP
BGP.as_path: 16652
BGP.next_hop: 38.104.127.2
BGP.local_pref: 500
198.252.153.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
Type: static unicast univ
BGP.origin: IGP
BGP.as_path: 16652
BGP.next_hop: 38.104.127.2
BGP.local_pref: 500
199.254.238.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
Type: static unicast univ
BGP.origin: IGP
BGP.as_path: 16652
BGP.next_hop: 38.104.127.2
BGP.local_pref: 500
show routes for specific IPs¶
bird> show route 161.0.161.0/24
161.0.161.0/24 via 38.104.127.1 on eth2 [cogent 16:56:44] * (500) [AS262237i]
via 38.104.127.1 on eth2 [swiftco 16:56:52 from 208.99.192.121] (100/?) [AS262237i]
bird>
This shows that we have two different possibilities for getting to this IP, one is via cogent and has a higher preference (500) and one is via swiftco, with a lower preference. The fact that the 38.104.127.1 ip is there for both of these should not fool you, that is just our ‘router id’ – for some reason people say that this should be the ip address, instead of a useful name, I dont know why.
bird> show route 198.252.153.0/24
198.252.153.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
bird>
This is showing one of our ip blocks, and that it is configured as a static route, and not available via any of our feeds (because we are announcing it, instead of receiving it)
show what routes are advertised to what BGP peer:¶
The following shows what routes we are advertising to the bgp peer ‘cogent’ (defined in the peers.conf):
bird> show route export cogent
204.13.164.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
198.252.153.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
199.254.238.0/24 unreachable [static_bgp 2014-10-07] ! (10000)
bird>
reloading¶
You can reload the configs and the filters, without restarting things completely. If you restart things completely, the routes are withdrawn and disappear from the internet, its much smoother to make changes without restarting.
Always check the configs before reloading:
bird> conf check
Reading configuration from /etc/bird/bird.conf
Configuration OK
If there is a syntax error, you will need to address that before reconfiguring, or things will not work right.
To reload the configuration do the following:
bird> configure
Reading configuration from /etc/bird/bird.conf
Reconfigured
Reloading can take a little bit to re-establish peers, and announce routes, so if you query the state immediately after reloading, you might not get what the quiescent state will be 30 seconds later, so keep that in mind.
route preferences¶
We had a situation where the route for one network was going out swiftco, because we are currently favoring routes out swiftco, but swiftco was returning that route to us, essentially causing a loop, and unreachability. To fix this, we needed to create an input filter for swiftco to set a lower preference for that network, so we would instead go out cogent. To do that, we first had to look at the swiftco peer definition to find what filters are applied (peers.conf):
### ASN 25700 Swiftco
protocol bgp swiftco from PEERS {
description "Swiftco";
source address 208.99.192.142;
neighbor 208.99.192.121 as 25700;
default bgp_local_pref 200;
multihop 255;
import filter bgp_in_swiftco;
export filter bgp_out;
}
This shows us that there are two filters, one input and one export, and their names are ‘bgp_in_swiftco’ and ‘bgp_out’. Those filters are defined in the filters.conf file. In order to change the preference locally for that specific network, we had to create a filter condition that matched that network, and if it did, it set the preference value for that netowork. This is what the final filter looked like:
filter bgp_in_swiftco
prefix as250_subnet;
{
if ! rt_import_all_asn(25700) then reject;
if (net.len > 16) then
bgp_local_pref=150;
else
bgp_local_pref=100;
if net ~ [ 193.227.234.0/23+ ]
then { bgp_local_pref=50; }
accept;
}
This sets the bgp_local_pref to be 50 if it matches that netblock (the + tells it to also match more specific). The number 50 was arbitrarily chosen, because it was lower than the default bgp_local_pref value, which is what the cogent peer is set to (100). This means that this network, if it comes in via the swiftco peer, will have a local preference that is lower (50) than the cogent preference (100), so it will favor the cogent preference.
This is before the filter was applied:
bird> show route for 193.227.234.0/23 all
193.227.234.0/23 via 208.99.192.141 on eth3 [swiftco 07:28:45 from 208.99.192.121] * (100/?) [AS250i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 25700 18530 209 1273 250
BGP.next_hop: 208.99.192.121
BGP.local_pref: 150
via 38.104.127.1 on eth10 [cogent 07:28:14] (100) [AS250i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 174 1273 250
BGP.next_hop: 38.104.127.1
BGP.med: 21040
BGP.local_pref: 100
BGP.community: (174,21000) (174,22013)
There are two possible routes listed above, one via swiftco, with a BGP.local_pref of 150, and one via cogent with a BGP.local_pref of 100. The higher 150 preference for swiftco meant that it would be sent out the swiftco peer. But because we are now applying this filter to that network, the swiftco preference will be lower.
This is the result after the filter was set:
bird> show route for 193.227.234.0/23 all
193.227.234.0/23 via 38.104.127.1 on eth10 [cogent 07:52:40] * (100) [AS250i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 174 1273 250
BGP.next_hop: 38.104.127.1
BGP.med: 21040
BGP.local_pref: 100
BGP.community: (174,21000) (174,22013)
via 208.99.192.141 on eth3 [swiftco 07:56:28 from 208.99.192.121] (100/?) [AS250i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 25700 18530 209 1273 250
BGP.next_hop: 208.99.192.121
BGP.local_pref: 50
bird>
As you can see above, again there are two possible routes, but now the first one shown at the top is via cogent, with the BGP.local_pref of 100 (same as before), and swiftco is at the bottom, with a BGP.local_pref of 50. The one above is the preferred route because it has the higher preference value.
peering¶
Peers are other providers that we establish a BGP session with to exchange routes and traffic. As of this writing we peer with two transit providers: Cogent and Isomedia. We also have other peers for other purposes, such as a peer sessions with the SIX route servers, and different peers that we’ve established sessions with through the SIX.
Because we are peering with the SIX routeservers, and we are announcing our routes there, the SIX participants will send traffic to us through that connection, instead of through our transit. When we first set this up, this moved 30mbit/sec immediately to our SIX cross-connect.
Peers are currently added/modified in the bird configuration file peers.conf and peers6.conf (for ipv4 and ipv6 respectively), and the peer ASN needs to also be added to the top of the filters.conf and filters6.conf in the PEER_ASNS variable. An example peer that we setup via the SIX was:
protocol bgp '206.81.80.10_6456' from SIX_STD { neighbor 206.81.80.10 as 6456; }
This sets up a BGP peer named ‘206.81.80.10_6456’ using the SIX_STD template that is just above it, and specifies the peer’s IP and ASN.
You can see the specific peer’s information in bird by doing this:
bird> show protocol all '206.81.80.10_6456'
name proto table state since info
206.81.80.10_6456 BGP master up 06:36:22 Established
Preference: 100
Input filter: (unnamed)
Output filter: bgp_out
Import limit: 100000
Action: restart
Routes: 3 imported, 3 exported, 0 preferred
Route change stats: received rejected filtered ignored accepted
Import updates: 6 0 0 3 3
Import withdraws: 557150 0 --- 557150 0
Export updates: 1771553 6 1771538 --- 9
Export withdraws: 7012 --- --- --- 1113817
BGP state: Established
Neighbor address: 206.81.80.10
Neighbor AS: 6456
Neighbor ID: 208.90.168.253
Neighbor caps: refresh AS4
Session: external AS4
Source address: 206.81.81.74
Route limit: 3/100000
Hold timer: 24/30
Keepalive timer: 1/10
bird>
and you can see what routes we are getting from that peer by doing this:
bird> show route all protocol '206.81.80.10_6456'
208.90.168.0/23 via 206.81.80.10 on eth11 [206.81.80.10_6456 06:36:26] (100) [AS6456i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 6456
BGP.next_hop: 206.81.80.10
BGP.med: 1
BGP.local_pref: 200
208.90.170.0/24 via 206.81.80.10 on eth11 [206.81.80.10_6456 06:36:26] (100) [AS6456i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 6456
BGP.next_hop: 206.81.80.10
BGP.med: 1
BGP.local_pref: 200
208.90.171.0/24 via 206.81.80.10 on eth11 [206.81.80.10_6456 06:36:26] (100) [AS6456i]
Type: BGP unicast univ
BGP.origin: IGP
BGP.as_path: 6456
BGP.next_hop: 206.81.80.10
BGP.med: 1
BGP.local_pref: 200
Scratch¶
echo “show route table t_spamd filter { if (65066,666) ~ bgp_community then accept; }”
need forwarding on
net.ipv4.ip_forward = 1
net.ipv6.conf.all.forwarding = 1
