Key takeaways

  • BGP runs over TCP port 179. Its hold time and keepalives detect a dead neighbor slowly. BFD (RFC 5880) runs a lightweight hello between the two routers and tells BGP within a fraction of a second.
  • RFC 7454 (BCP 194) is the baseline for safe BGP: filter bogons and over-specific prefixes, limit the number of prefixes per peer, check the AS path, and protect the session itself.
  • RPKI origin validation (RFC 6811) marks each route Valid, Invalid or NotFound by comparing the prefix and its origin AS with signed ROAs. It protects against accidental and simple malicious origin mistakes. It does not validate the whole path.
  • BGPsec (RFC 8205) validates the AS path with signatures. It is a separate protocol from origin validation, requires both peers to negotiate it and needs router keys. Most operators today deploy origin validation, not BGPsec.
  • Communities, including the 12-byte large community (RFC 8092), let you tag routes by where they were learned and what policy applies. Define them once and write policy against the tags, not against individual peers.
  • A route validation cache (an RPKI-to-router, or RTR, server) is a dependency. Run more than one, and know what your router does when it loses them.

What the edge has to do

The edge of a network is where its address space meets the rest of the internet. A router there announces your prefixes to upstream carriers and exchange points, receives other networks' routes and picks the best path for each destination. The protocol for that is the Border Gateway Protocol, version 4, defined in RFC 4271. BGP runs over TCP port 179, exchanges OPEN, UPDATE, NOTIFICATION and KEEPALIVE messages, and moves through the states Idle, Connect, Active, OpenSent, OpenConfirm and Established.

An edge that is not carefully configured can hurt other people. A leaked route can pull traffic into the wrong network. An unfiltered customer can announce someone else's prefix. A single flapping session can burn CPU on every router that hears about it. Almost everything in this article is about preventing those outcomes, and about recovering quickly when a link or router fails.

Why BIRD 2

BIRD is an open-source routing daemon that runs on Linux and BSD. Version 2 handles IPv4 and IPv6 in one daemon, with a protocol for BGP, a protocol for BFD, a protocol for RPKI, and a filter language for policy. A server with fast network cards can act as a router and a route server, and a pair of them can serve as a small provider's edge or as a data center's internet gateway. Large forwarding rates on a commodity server are possible, but the packet path (kernel, XDP or a hardware router) is a separate decision from the routing daemon. BIRD computes routes and installs them; something else forwards packets.

A baseline from RFC 7454

RFC 7454 is BCP 194, titled BGP Operations and Security. It is the checklist most edge configurations should begin with. Its main recommendations are:

  • Session protection. Prefer TCP-AO (RFC 5925) where supported; TCP MD5 (RFC 2385) is still widely used. Use the Generalized TTL Security Mechanism (GTSM, RFC 5082) on directly connected peers, so packets must arrive with TTL 255. Block spoofed source addresses at the edge. Restrict TCP 179 to known neighbors.
  • Prefix filtering. Discard bogons and special-purpose prefixes on internet peerings. Do not accept anything longer than /24 for IPv4 or /48 for IPv6. Filter your own prefixes inbound, the exchange point LAN, and the default route unless you want it. Accept from a customer only the prefixes registered to that customer.
  • Maximum prefixes. Set a limit per session, below the full table for peers and above it for full-table upstreams. Use two thresholds where available: the lower one logs and the higher one drops the session.
  • AS path filtering. Require that the first AS in the path is the peer's AS (except on route servers), reject paths that contain your own AS, drop private AS numbers from non-customers and limit excessive prepending.
  • Communities hygiene. Scrub inbound communities that use your own AS number in the high-order bits, so a neighbor cannot trigger your internal policy.

A BIRD import filter implements these rules in a few lines each. The language is declarative, with accept and reject statements, and every filter must end in one or the other. For external BGP, BIRD requires you to write the import and export policy explicitly inside the channel, so a missing filter fails safe instead of silently accepting everything.

A skeleton BIRD 2 session

The BIRD documentation shows the shape of a BGP protocol block. A session to an upstream might look like this, with documentation addresses and AS numbers:

  • protocol bgp upstream1 { local 198.51.100.14 as 65000; neighbor 198.51.100.130 as 64496; ttl security yes; password "<secret>"; graceful restart aware; bfd graceful; ipv4 { import filter ebgp_in; export filter ebgp_out; }; }
  • The options used are documented BIRD 2 keywords: local and neighbor name the two ends, ttl security turns on GTSM, password sets the TCP MD5 key, graceful restart aware is the default, and bfd graceful enables BFD for the session and triggers graceful restart if BFD declares failure.
  • If the neighbor is not directly connected, multihop sets the hop count, and GTSM is then less effective, because anyone within the TTL range could spoof a packet. Prefer directly connected sessions at the edge.

RPKI origin validation

The oldest weakness of BGP is that it takes announcements on trust. RPKI adds signed statements. A Route Origin Authorization (ROA) says that a given AS is authorized to originate a prefix, up to a maximum length. A relying-party tool validates the signatures and produces a list of Validated ROA Payloads, each with a prefix, a maximum length and an origin AS. RFC 6811 then defines how a router uses them.

The algorithm is short. A payload covers a route if its prefix is equal to or shorter than the route's prefix and matches its bits. It matches if it covers the route, the route's length is no longer than the payload's maximum length, and the origin AS is the same. The route's state is Valid if any payload matches, Invalid if some payload covers the route but none matches, and NotFound if none covers it. A payload for AS 0 never matches. The origin is the last AS in the AS path.

RFC 6811 is careful about policy. A router should not drop a route from its tables merely because of the state unless the operator configures it that way, and treating Invalid routes in the decision process is a local choice to be handled with great care. RFC 7454 recommends a conservative stance for internet peerings: accept Valid, discard Invalid, and accept NotFound at lower preference. Most operators follow that, and many publish ROAs for their own prefixes first so their customers are not caught out.

BIRD receives the payloads from a validator through the RPKI-to-Router protocol, RFC 8210, which uses TCP port 323 unprotected or 324 with TLS (the BIRD protocol can also use SSH). In BIRD 2 the rpki protocol feeds a roa4 or roa6 table, and the filter calls roa_check on the route, which returns ROA_VALID, ROA_INVALID or ROA_UNKNOWN. A typical import filter rejects ROA_INVALID and lets the rest continue. The protocol has refresh, retry and expire timers; the RFC defaults are 3,600, 600 and 7,200 seconds, and the cache must not let data outlive the expire interval. The practical lesson: run at least two validators in different places, monitor them, and know whether your router fails open or closed when they disappear.

What BGPsec is, and what it is not

BGPsec, RFC 8205, is a different protocol. It adds a BGPsec_PATH attribute that replaces the AS path and carries a signature from each AS that forwarded the update, so a validating router gains assurance that each AS on the path authorized the announcement. Origin validation checks the first hop; BGPsec checks the rest. The RFC says it supplements origin validation, and that both together are meant to prevent a wide range of route hijacks.

BGPsec asks more of its users. A speaker that sends signed updates to external peers needs a private key tied to an RPKI router certificate for its AS. Both sides must negotiate the capability and support four-byte AS numbers. If any AS in the path does not support it, the update falls back to an unsigned one and assurance ends at that point. For these reasons, and because it protects the control plane only, the practical edge today is origin validation plus RFC 7454 filtering. Mention BGPsec as something to watch and plan for, not as a feature you switch on.

Communities: the language of policy

A BGP community is a tag attached to a route. Classic communities pack two 16‑bit values into 32 bits, which does not fit a four-byte AS number. The large community of RFC 8092 is 12 bytes: a four-byte global administrator, normally an AS number, and two four-byte local data fields, written as three decimal numbers separated by colons. BIRD supports large communities as a native type in filters.

A good community plan is small. Tag every route on import with where it was learned (customer, peer, upstream, internal) and in which region. Use further tags to say what you want to happen: prefer this path, do not announce to that peer, prepend twice to this region. Then write export policy purely against the tags. Publish the list for your customers, and scrub inbound copies of the communities you own, as RFC 7454 recommends.

One community deserves its own mention. RFC 7999 defines BLACKHOLE (65535:666) as a signal to drop traffic to a prefix, normally a /32 or /128 for the victim of a flood. The receiver should honor it only for prefixes the neighbor may announce and only where it has agreed to. We discuss it in our companion article on surviving a flood.

Fast failover with BFD

BGP detects a dead neighbor through its hold timer. RFC 4271 says a negotiated hold time must be zero or at least three seconds, and that a reasonable keepalive interval is one third of the hold time. Tuning it very low costs CPU and can cause flaps, so many sessions rely on tens of seconds. That is too slow when a link silently fails without dropping carrier.

Bidirectional Forwarding Detection, RFC 5880, is a separate and lightweight protocol that detects a fault in the path between two forwarding engines independent of the routing protocol. In asynchronous mode both systems send periodic control packets. The detection time is the remote side's detect multiplier times the agreed transmit interval, and a session goes down when no packet arrives in that window. Single-hop BFD for IP (RFC 5881) uses UDP destination port 3784 and requires a TTL of 255 on received packets. So at 50 ms intervals with a multiplier of three, a failure is noticed in about 150 ms, subject to what the hardware and links can sustain.

In BIRD 2 you declare a bfd protocol, with per-interface min rx interval, min tx interval and multiplier values, and then turn it on per BGP session with the bfd option. When BFD declares the session down, BIRD tears down the BGP session at once and withdraws the routes. The remaining router converges to the alternative path. Two cautions: choose intervals the platform can keep up with under load, because a router that misses its own BFD packets will cause the very failure it is meant to detect, and test failover under real traffic.

Operating practices

  • Version control the configuration, and generate filters from data (prefix lists from IRR or from your own database) rather than editing by hand.
  • Announce only what you have to. Aggregate where you can, and publish ROAs that match what you announce.
  • Use graceful shutdown or maintenance communities before planned work, so traffic moves off a router before it is rebooted.
  • Monitor session state, prefix counts, RPKI cache health and BFD flaps. Alert on a session that sits in Active, and on any rapid change in prefix counts.
  • Write a runbook for the three bad days: a leaked route, a hijack of your prefix and a lost RPKI cache.
  • Keep two independent paths to the internet, with different carriers where possible, and test the loss of each.

Our team designs and operates edge routing for organizations that need more than a managed circuit. We build it from the standards above and validate it with failure drills before it carries production traffic. Related reading: MPLS, SD-WAN or EVPN: Choosing a Backbone for a Multi-Site Company; DNS Security: DNSSEC, Response Policy Zones and Encrypted DNS.

Sources

This article is general technical information, not a configuration for your network. Test every filter and timer in a lab and in a maintenance window before production. The example addresses are documentation addresses. The BFD detection example (50 ms times three) is an illustration of the formula in RFC 5880, not a performance guarantee. Last reviewed .

Keep reading

Need help with this?

Cloud cost review and management, without compromising on quality or security.

Cloud Management