Thomas' ramblings

Generating NAPTR and SRV record for TINYDNS

Mon, 30 Jun 2008 15:51:52 GMT

TINYDNS is a good DNS server, however tinydns-data is missing some builtin syntax for the generation of NAPTR and SRV record.

Anders Brownworth wrote a nice web page on which generate those record using the tinydns generic record syntax here.

However I needed to be able to generate those record for the configuration of IENUM (technically ENUM on private DNS) from some of our python code.
As a result I wrote the following "library" to generate some domain SIP NAPTR and SRV records.

Hopefully it may save someone the time to reverse engineering Anders' page output.

Rambling

Mon, 30 Jun 2008 13:05:00 GMT

Once uppon a time, I used to ramble a lot about various ISP/technical related matters (mainly Phorm and ISP 'hidden' traffic shaping). I removed most of the documents as I was not updating them often enough to keep them accurate and provided a summary on traffic shaping and Phorm.

The real cost of P2P

Mon, 30 Jun 2008 13:01:00 GMT

Should you read Slashdot, you must have already seen its readers complaining about their ISP traffic shaping policies.
When working in the ISP industry it is painful to see the lack of understanding those 'techies' are displaying.

In the UK, if anything ISPs are guilty of bad advertising misleading customers with 'up to' speeds and obscure fair usage policies and trying to market their product on price instead of quality (but Internet is a commodity market nowdays, so it is to be expected)

Customers should be clearly told that DSL product sold are contended. Previously dialup products were as well, but the impact with dialup was much more noticeable with the inability to get online.

The recent increase in content (video even more than P2P) has recently caused many of them to realise that they had oversubscribed their infrastructure to the point they could not deliver to their customers what they came to expect.
Once down to the wall, ISPs had only a few options :
* raise price to reflect the cost of running the service at a low contention (and we all know that it is impossible)
* apply traffic policing globally (everyone is slowed down the same way to modem speed).
* apply targeted traffic policy (P2P users here you are)

As it is hard to tell a customer, who may cancel its contract returning a then useless free router, that he can no longer have fast email and web surfing, the path of least resistance is to throttle P2P traffic which is an important part (but not all) of an ISP traffic, freeing capacity for other services and allowing to delay infrastructure upgrade.
(The cost of implementing traffic shaping is recovered if it allows to delay a network upgrade if only for a month!)

For information, an ISP for a DSL service can be simplified as:
* the 'last mile' cost from the home to the exchange
* the cost of the space used, power consumed and hardware located at the exchange
* the cost of moving the traffic within the country (fiber, etc.)
* the cost of the space used, power consumed and hardware located at national point of presence
* the cost of moving the traffic to other ISPs
* the cost of supporting the customer (ie: taking unrelated calls about their virus or other issues)
* the cost of collection the client payment
* all other generic business

For quite few small/medium ISPs, the transit cost (the cost an ISP will pay for another bigger ISP to take its traffic somewhere worldwide) is more than the income that the customer provides. Most ISPs are making a loss trying to become big enough to be acquired.
P2P being notably known to not really care about locality, one can see why it is the target of shaping (with the fact that the biggest torrent are often providing copyrighted material for which end users may or may not have a license to see/use).

In that context it is not surprising that the industry is facing issues and trying to find more income streams (see my rant on Phorm).

Software

Mon, 30 Jun 2008 11:28:00 GMT

All the documents related to software I wrote are tagged with 'Software', you can find them using the search feature or the 'Tags' tab.

Network

Mon, 30 Jun 2008 11:27:00 GMT

All the documents related to networking are tagged with 'Network', you can find them using the search feature or the 'Tags' tab.

PageTemplate

Sun, 29 Jun 2008 20:11:00 GMT

<div class='header' macro="gradient vert #5c4894 #6b69ad">
    <div>
       <span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>&nbsp;
       <span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
    </div>
    <div id='topMenu'>
       <span refresh='content' tiddler='MainMenu'></span>
    </div>
</div> 
<div id='sidebar'>
	<div id='sidebarOptions' refresh='content' tiddler='SideBarOptions'></div>
	<div id='sidebarTabs' refresh='content' force='true' tiddler='SideBarTabs'></div>
</div>
<div id='displayArea'>
	<div id='messageArea'></div>
	<div id='tiddlerDisplay'></div>
</div>

StyleSheet

Sun, 29 Jun 2008 20:07:00 GMT

.headerForeground { display: none;}
#sidebar {width: 170px; background: #efefef;border-left: solid 2px #b8b9c7;border-top: solid 2px #d7d8e8;}
#sidebarTabs .tabContents {width: 158px; background: #eae9ee;font-weight: bold; color: #333 ;}
#sidebarOptions input { border: solid 2px #b8b9c7; }
#sidebarOptions .sliderPanel { background: #eee;}
#sidebarOptions a {;border: none;}
#sidebarOptions .sliderPanel a {border: none;color: #5c4894;}
#displayArea {background: #fff;margin: 1em 15.7em 0em 1em;border-left: solid 2px #b8b9c7;}
.viewer {line-height: 1.4em;padding-bottom: 1em;border-bottom:solid 1px #b8b9c7;}
.viewer th, thead td {background: #5d4b97;border: 1px solid #666;color: #fff;}
.title {color: #000}
h1,h2,h3,h4,h5 {color: #fff;background: #6b69ad;}
a{ color: #700126;}
a:hover{ background: #6b69ad; color: #fff;font-weight: bold;}
.externalLink {	text-decoration: underline; color: #000083;}
body {	background: #d7d8e8;}
.popup { background: #6b69ad; border: 1px solid #04b;}
.popup li a:hover {background: #d7d8e8;color: #000;border: none;}
.popup li.disabled {color: #000;}
.button:hover {color: #fff;background: #6b69ad;
	border: 1px solid #d7d8e8;}
#topMenu { background: transparent; padding: 6px;margin-left: -5px;border-bottom: solid 3px  #5c4894;}
#topMenu .button,  #topMenu .tiddlyLink, tiddlyLinkExisting, #topMenu .externalLink
{
	color: #fff;
	text-align: center;
	font-weight: bold;
	font-size: 1.1em;
	text-decoration: none;
	letter-spacing: 1.5px;
	background: transparent;
	border-right: solid 1px #fff;
        padding: 5px 15px 8px 15px;
}
#topMenu a:hover {
	color: #700126;
	background: #d7d8e8;
}
#topMenu br {display: none; padding-right: 1em;}
#topMenu span .tiddlyLinkNonExisting {font-style:normal;}

Topology

Sun, 29 Jun 2008 18:44:00 GMT

The bgp articles on this site are using based on the following virtual network. The router are owned by an ISP with its own AS number. The IGP is EIGRP as it is the protocol I am most familiar with at the time of writing (it could be OSPF or IS-IS).

Servers

The following server are present on the network:
10.0.0.1 Primary caching DNS
10.1.0.1 Secondary caching DNS
10.2.3.200 SNMP Monitoring station
10.2.3.201 SYSLOG loging server
10.2.3.202 NTP server
10.2.3.205 Netflow monitoring server

IP Range

The ISP internal range is:
10.0.0.0/16 AS 65200
10.1.0.0/16 AS 65200
10.2.3.0/24 AS 65200

The ISP provides transit to:
192.168.0.0/24 AS 65350
192.168.1.0/24 AS 65360

The Internet Exchange

The exchange Information is running a dual ring topology with a separate /23 for each
172.16.0.0/23 AS 65400
172.16.2.0/23 AS 65400

Internal routing

It is assumed that the 192.168.0.0/24 and 192.168.1.0/24 are available through the IGP. The client eBGP connection is configured to pass through another internal routers (multi-hop), which will have a directly connected interface or a static route to the networks advertised.

No encryption is used between BGP peers, no access list is in place to protect EIGRP, BGP and NTP traffic to the router.

The local preference used are beetween 100 and 200. To calculate the RIPE preference, apply the following rule : ripe_pref = 200 - local_pref

Home

Sun, 29 Jun 2008 18:33:00 GMT

Software related stuff
Network related stuff
About this site and me

Should you want to contact me feel free to email me here.
My public PGP key is here.

Should you want to link to this site please use http://thomas.mangin.com/ as hostname

GettingStarted

Sun, 29 Jun 2008 18:31:00 GMT

To get started with this blank TiddlyWiki, you'll need to modify the following tiddlers:

SiteTitle & SiteSubtitle: The title and subtitle of the site, as shown above (after saving, they will also appear in the browser title bar)
MainMenu: The menu (usually on the left)
DefaultTiddlers: Contains the names of the tiddlers that you want to appear when the TiddlyWiki is opened

You'll also need to enter your username for signing your edits:

DefaultTiddlers

Sun, 29 Jun 2008 18:24:00 GMT

Welcome
News
About

MainMenu

Sun, 29 Jun 2008 18:22:00 GMT

Welcome
Network
Software
Rambling

SiteSubtitle

Sun, 29 Jun 2008 18:20:00 GMT

a site with no purpose

SiteTitle

Sun, 29 Jun 2008 18:19:00 GMT

Thomas' ramblings

SiteUrl

Sun, 29 Jun 2008 18:16:00 GMT

http://thomas.mangin.com/

Phorm, the logical conclusion to legal pressure on ISP

Fri, 29 Feb 2008 11:33:00 GMT

Up to recently, ISP felt that they had the same status as traditional telcommunication provider and were protected from prosecution for the traffic going through their network. It was then none of their business to police the information flowing through their network.

The situation became hazier when BT decided to deploy cleanfeed. Up to that point ISP had been transproxying web traffic in order to cache the web page requested and save on bandwidth cost but had never actively interfered with the data passing through their network.

More recently threat of legislation pushed by the IFPI, children protection lobby, and government (all ignoring that transproxying can be easily evaded) seems to be changing the landscape for ISP, which are now under increased pressure to police their traffic for the benefit of who can afford to lobby them.

Deploying large scale filtering/transproxying solution is expensive, and with little chance of seeing the cost paid the either the end user or the legislator, It is only natural for ISP to seek some kind of form or remineration of the cost of deploying such possibly soon legally required solutions.

In that context it is not that strange to see the UK largest ISP sell their customer web traffic (not protect by any data protection law) to an organisation selling targeted advertising.

Up to now, advertiser had to rely on cookies to track surfing habit, making it possible for customers to protect their privacy (refusing them or using anomymisers).

With this new system (described here) our average UK broadband users can only hope that the ISP marketing firm will honor its promise to not monitor their traffic.

The most interesting part seems to be that even once 'unsubscribed' the traffic may still go through the advertiser 'anomyser proxies'.
One can only wonder if those proxies role will not block cookies from competitors giving Phorm a quasi monopoly for advertising in the UK.

Multiple Recipients for Postfix Access Policy Delegation

Fri, 16 Nov 2007 10:12:00 GMT

Use the patch provided here at your own risk : do not use if you are not able to understand the code provided

Before using this patch, you may want to read this thread on the postfix-user mailing list where I was told:

that I am ill advised to want such a patch in postfix as its approach is fundamentally flawed
that this patch is too resource intensive

In order to address the last point, I made sure that :

the feature is turned off by default
the maximum among of memory available to the feature can be set.

With the default values :
smtpd_client_connection_count_limit (default: 50)
smtpd_recipient_limit (default: 1000)
line_length_limit (default: 2048)

The worse case memory utilisation for the feature is around 2Mb per smtpd instance which is 40Mb with the default settings - which are exceptionally large for the recipient limit. Limiting mails to 50 recipients makes the worse case overhead per smtpd 100kb.

40 Mb is indeed a lot for an old machine but on recent hardware it will not even be noticed (and this memory will only be allocated if the mails received have lots of recipients).

The other way to get all the recipients of a mail would be to track the "recipient" sent to the policy server at each RCPT using the "instance" attribute and use the result at the DATA state.

With this approach the policy server need :

to be called at each RCPT (and not only at DATA)
keep track of the recipients for each mail
to perform some cleaning should the connection close between the RCPT and DATA state

The patch provides two new configuration options:

a boolean : access_delegation_recipients, which need to be turned on to use the feature
an integer : smtpd_recipients_length_limit, which limit the among of memory the list of recipients can take, it is set to zero by default meaning that no limitation will be performed. Should its value be under "line_length_limit", the value will be changed at run time to this default.

It changes the SMTPD POLICY Protocol adding a line starting with "recipients=". The key contains a "\r" separated list of the mail recipients (or the single recipient, exactly as the recipient key).
The list is only set during the ~DATA and END_OF_DATA state and only if the lenght of the string is under the value set in smtpd_recipients_lenght_limit.

This patch/feature _is_ useful for :

boucing spam to a list of forged inexistent email addresses (especially when your MX and storage servers are not on the same machines).
to allow per domain policies, ie per domain white-listing, etc.
you tell me

You can download the fourth version (released the 26th of November 2007) of this patch
here for postfix 2.6 20071111

I have updated the patch to apply cleanly on a more recent version of postfix
here for postfix 2.6 20080201 (which applies cleanly on postfix-2.5.1-rc1)

Should you have downloaded any previous version, please update as the third contain a memory leak which cause the memory utilisation to be up to two times what it should and any version before should simply not be used.

Juniper Peering Router Configuration Example

Mon, 22 Oct 2007 14:28:00 GMT

Should you be looking at using a Juniper router for an EBGP connection, I hope the following Junos configuration will prove useful.

I have tried to keep it short removing community based firewalling (as you can read about it here, class-of-service, logical-routers, event-options, snmp, and god knows what more to try to keep the resulting configuration short.

A basic ISIS section was left to show how to routes can be originated on the router itself.

A skeleton of firewall filters was left to give a taste of what can be done to protect the core from spoofed traffic, ICMP flooding,etc.
Should it be something of interrest please consider reading The Junos secure template

Route damping was left in but is inactive as recomended by ripe-378 which obsoletes ripe-229, ripe-210 and ripe-178

A lot is still present tho like community controlled route announcement, communitiy triggered route blackholing and bgp leak mitigation using as-path.

I am pretty sure that in the fury of cut, paste and replace done, I must have broken enough of the configuration to make it unadviseable to try to use it "as it" but it should give you a good head start if you never done it before.

The configuration is not yet commented (or split in part) but I will try to fix this at some point (as well as fix the formating which this wiki likes to remove)

Use at your own risk and feel free to let me know if something is wrong (I never had the opportunity to test the bgp triggered route black blackhole yet).

version 8.2R3.6;

/* Template for all the interface on the router */
groups {
 peering-interface {
 interfaces {
 <*> {
 unit <*> {
 family inet {
 filter {
 input external-incoming-peer;
 }
 }
 }
 }
 }
 }
 physical-interface {
 interfaces {
 traceoptions {
 file interfaces size 1m files 5;
 flag change-events;
 }
 <ge-*> {
 traps;
 vlan-tagging;
 link-mode full-duplex;
 gigether-options {
 flow-control;
 }
 unit <*> {
 family inet {
 no-redirects;
 }
 }
 }
 }
 }
 core-interface {
 interfaces {
 <*> {
 unit <*> {
 family inet {
 no-redirects;
 }
 }
 }
 }
 }
 transit-interface {
 interfaces {
 <*> {
 unit <*> {
 family inet {
 rpf-check {
 mode loose;
 }
 filter {
 input-list [ sample-netflow external-incoming-transit ];
 }
 }
 }
 }
 }
 }
 customer-interface {
 interfaces {
 <*> {
 unit <*> {
 family inet {
 rpf-check {
 mode loose;
 }
 filter {
 input external-incoming-customer;
 }
 }
 }
 }
 }
 }
}

/* System Configuration */
system {
 host-name m7i;
 domain-name business.net.uk;
 domain-search [ business.net.uk ];
 time-zone Europe/London;
 no-redirects;
 authentication-order tacplus;
 location {
 country-code UK;
 postal-code "";
 building "Telehouse";
 rack 123;
 }
 ports {
 console type vt100;
 }
 root-authentication {
 encrypted-password "$"; ## SECRET-DATA
 }
 name-server {
 ip;
 ip;
 }
 tacplus-server {
 ip {
 secret "$"; ## SECRET-DATA
 timeout 5;
 }
 }
 accounting {
 events [ login change-log interactive-commands ];
 destination {
 tacplus {
 server {
 ip secret "$"; ## SECRET-DATA
 }
 }
 }
 }
 scripts {
 /* See juniper.cluepon.net */
 }
 login {
 message "******************************************************************************\n NOTICE TO USERS\n\nThis equipment is for authorized use only. Users (authorized or unauthorized)\nhave no explicit or implicit expectation of privacy.\n\nAny or all uses of this system and all files on this system may be intercepted,\nmonitored, recorded, copied, audited, inspected, and disclosed to authorized\nsite and law enforcement personnel.\n\nBy using this system, the user consents to such interception, monitoring,\nrecording, copying, auditing, inspection, and disclosure at the discretion of\nauthorized site.\n\nUnauthorized or improper use of this system may result in administrative\ndisciplinary action and civil and criminal penalties. By continuing to use\nthis system you indicate your awareness of and consent to these terms and\nconditions of use.\n\nLOG OFF NOW if you do not agree to the conditions stated in this warning.\n\nBusiness Limited - noc@business.co.uk - +44 \n*****************************************************************************\n\n";
 class administrator {
 idle-timeout 60;
 permissions all;
 }
 class linx {
 permissions [ field interface routing trace view view-configuration ];
 }
 user admin {
 full-name "Admin";
 uid 1000;
 class administrator;
 authentication {
 encrypted-password "$"; ## SECRET-DATA
 }
 }
 user linx {
 full-name "Linx Staff Access";
 uid 1001;
 class linx;
 authentication {
 encrypted-password "$"; ## SECRET-DATA
 }
 }
 }
 static-host-mapping {
 tacplus inet ip;
 syslog inet ip;
 localhost inet 127.0.0.1;
 m7i-4.u3.tcw.uk {
 inet ip;
 sysid 0822.1900.0068;
 }
 }
 services {
 ssh {
 root-login deny-password;
 protocol-version v2;
 connection-limit 5;
 rate-limit 10;
 }
 telnet {
 connection-limit 5;
 rate-limit 10;
 }
 }
 syslog {
 archive size 1m files 10;
 user * {
 any error;
 }
 host ip {
 /* none, info, notice, warning, error, critical, alert, emmergency */
 any notice;
 facility-override local6;
 }
 file messages {
 any notice;
 authorization info;
 }
 file interactive-commands {
 interactive-commands any;
 }
 file system {
 daemon any;
 kernel any;
 }
 file firewall {
 firewall any;
 }
 file security {
 authorization any;
 interactive-commands any;
 }
 file user-comand {
 interactive-commands info;
 }
 console {
 any error;
 }
 source-address ip;
 }
 no-compress-configuration-files;
 archival {
 configuration {
 transfer-on-commit;
 archive-sites {
 "ftp://user:pass@ip/text/router-name/";
 }
 }
 }
 ntp {
 boot-server ip;
 server ip;
 server ip;
 }
}

/* Prevent an alarm if nothing is plugged on the console */
chassis {
 no-source-route;
 alarm {
 management-ethernet {
 link-down ignore;
 }
 }
}

/* Interfaces Configuration */
interfaces {
 apply-groups physical-interface;
 ge-0/3/0 {
 description "LAN";
 unit A-VLAN {
 apply-groups core-interface;
 description "Internal Switches";
 vlan-id THE-VLAN-NUMBER;
 family inet {
 address range/netmask;
 }
 }
 unit A-VLAN {
 apply-groups core-interface;
 description "to Elsewhere";
 bandwidth 40;
 vlan-id THE-VLAN-NUMBER;
 family inet {
 filter {
 /* Filter ddos on output as it seems to cause issue on input on internal interface */
 output ddos-protect;
 }
 address ip/30;
 }
 family iso;
 }
 }
 ge-1/3/0 {
 description "Upstream Interface";
 unit 123 {
 apply-groups peering-interface;
 description Linx;
 vlan-id THE-VLAN-NUMBER;
 family inet {
 address 195.66.224.---/23;
 }
 }
 }
 fxp0 {
 description "Management Interface";
 unit 0 {
 family inet {
 no-redirects;
 filter {
 input protect-management;
 }
 }
 }
 }
 lo0 {
 unit 0 {
 description Loopback;
 family inet {
 no-redirects;
 address ip/32;
 }
 family iso {
 address 49.0001.0822.1900.0071.00;
 }
 }
 }
}

forwarding-options {
 sampling {
 input {
 family inet {
 rate 1000;
 inactive: run-length 4;
 max-packets-per-second 7000;
 }
 }
 output {
 cflowd ip {
 port 2055;
 source-address ip;
 version 8;
 no-local-dump;
 autonomous-system-type origin;
 aggregation {
 autonomous-system;
 }
 }
 }
 }
 hash-key {
 family inet {
 layer-4;
 }
 }
}

routing-options {
 options {
 syslog {
 level debug;
 }
 }
 graceful-restart;
 interface-routes {
 rib-group inet if-rib;
 }
 /* Black Hole route */
 route 127.0.0.2/32 {
 discard;
 retain;
 no-readvertise;
 }
 aggregate {
 route your-network/range {
 community 54321:54321;
 as-path {
 origin igp;
 }
 }
 }
 rib-groups {
 if-rib {
 import-rib [ inet.0 inet.2 ];
 }
 isis-rib {
 export-rib inet.0;
 import-rib [ inet.0 inet.2 ];
 }
 mcast-rib {
 export-rib inet.2;
 import-rib inet.2;
 }
 }
 router-id ip;
 autonomous-system 54321;
 forwarding-table {
 export [ load-balancing ];
 unicast-reverse-path feasible-paths;
 }
}

protocols {
 bgp {
 path-selection always-compare-med;
 log-updown;
 inactive: damping;
 graceful-restart;
 group ibgp {
 type internal;
 traceoptions {
 file bgp-ibgp size 1m files 5;
 }
 local-address ip;
 import blackhole;
 authentication-key "$"; ## SECRET-DATA
 export [ originate-community originate-customer export-ibgp next-hop-self ];
 peer-as 54321;
 neighbor ip;
 }
 group transit {
 type external;
 local-preference 75;
 remove-private;
 neighbor IP {
 inactive: traceoptions {
 file bgp-transit1 size 1m files 5;
 }
 description "ANY | Transit 1 | myfault@transit1 |";
 local-address ip;
 import [ no-ix no-bogons no-small-prefixes tag-transit tag-transit1 damping local-preference-transit no-community-import ];
 export [ originate-community originate-customer no-transit no-small-prefixes export-transit export-transit1 no-community-export next-hop-self ];
 peer-as 1234;
 }
 }
 group linx-collector {
 type external;
 inactive: traceoptions {
 file bgp-linx-collector size 1m files 5;
 flag all;
 }
 description "Linx Route Collector";
 local-preference 150;
 local-address 195.66.224.---;
 import [ no-ix no-bogons no-small-prefixes no-leak tag-peering tag-linx damping local-preference-peer no-community-import ];
 export [ originate-community originate-customer no-transit no-small-prefixes export-peering export-linx no-community-export next-hop-self ];
 remove-private;
 neighbor 195.66.224.254 {
 /* See cluepon.juniper.net for the op script which transform this */
 apply-macro inet {
 prefix-limit 500;
 }
 description "NOT ANY | Linx Route Collector | |";
 family inet {
 unicast {
 prefix-limit {
 maximum 500;
 }
 }
 }
 authentication-key "$"; ## SECRET-DATA
 peer-as 5459;
 }
 }
 group linx-route-server {
 type external;
 inactive: traceoptions {
 file bgp-linx-rs size 1m files 5;
 flag all;
 }
 description "LINX Route Servers";
 local-preference 125;
 local-address 195.66.224.---;
 import [ no-ix no-bogons no-small-prefixes no-leak tag-peering tag-linx damping no-community-import ];
 export [ originate-community originate-customer no-transit no-small-prefixes export-peering export-linx no-community-export next-hop-self ];
 remove-private;
 neighbor 195.66.225.230 {
 apply-macro inet {
 prefix-limit 19534;
 }
 description "ANY | Linx route server | | AS-EXA";
 authentication-key "$"; ## SECRET-DATA
 peer-as 8714;
 }
 neighbor 195.66.225.231 {
 apply-macro inet {
 prefix-limit 19229;
 }
 description "ANY | Linx route server | | AS-EXA";
 authentication-key "$"; ## SECRET-DATA
 peer-as 8714;
 }
 }
 group renesys {
 type external;
 inactive: traceoptions {
 file bgp-renesys size 1m files 5;
 }
 description "A full routing table for Renesys at Linx";
 local-address 195.66.224.---;
 import deny-all;
 export [ originate-community originate-customer no-small-prefixes no-community-export next-hop-self ];
 remove-private;
 neighbor 195.66.225.--- {
 peer-as 64---;
 }
 }
 group linx {
 type external;
 traceoptions {
 file bgp-linx size 1m files 5;
 flag state;
 flag route;
 flag general;
 flag normal;
 flag open;
 flag policy;
 }
 local-preference 150;
 local-address 195.66.224.---;
 import [ no-ix no-bogons no-small-prefixes no-leak tag-peering tag-linx damping local-preference-peer no-community-import ];
 export [ originate-community originate-customer no-transit no-small-prefixes export-peering export-linx no-community-export next-hop-self ];
 remove-private;
 neighbor ip {
 apply-macro inet {
 prefix-limit 500;
 }
 description "AS-MACRO | Name | noc@isp |";
 peer-as 65555;
 }
 }
 }
}

protocols {
 isis {
 traceoptions {
 file isis size 1m files 5;
 flag normal;
 flag error;
 }
 export static-to-isis;
 loose-authentication-check;
 no-ipv6-routing;
 rib-group inet isis-rib;
 level 1 {
 authentication-key "$"; ## SECRET-DATA
 authentication-type simple; ## SECRET-DATA
 }
 level 2 {
 authentication-key "$"; ## SECRET-DATA
 authentication-type simple; ## SECRET-DATA
 }
 interface ge-0/3/0.VLAN-1 {
 lsp-interval 33;
 checksum;
 level 1 {
 hello-interval 10;
 hold-time 30;
 }
 level 2 {
 hello-interval 10;
 hold-time 30;
 }
 }
 interface ge-1/3/0.VLAN-2 {
 passive;
 }
 interface all {
 level 1 disable;
 }
 interface fxp0.0 {
 disable;
 }
 interface lo0.0 {
 passive;
 }
 }
}

policy-options {
 prefix-list root-servers {
 /* Add routes servers here : see www.cymru.com */
 }
 prefix-list rfc1918-reserved {
 /* RFC 1918 addresses */
 10.0.0.0/8;
 172.16.0.0/12;
 192.168.0.0/16;
 }
 prefix-list protected-address {
 /* IP ADDRESS The internet should not be able to reach within your network */
 }
 prefix-list business-external {
 /* Part of your ip space used for interconnect to customers (so to be allowed in the network) */
 }
 prefix-list ssh-address {
 /* What IPs can SSH/telnet in */
 }
 prefix-list bgp-address {
 /* Your BGP peers */
 }
 prefix-list dns-address {
 /* Your DNS servers */
 }
 prefix-list ntp-address {
 /* Your NTP servers */
 }
 prefix-list snmp-address {
 /* Your SNMP server - pulling and trap .. */
 }
 prefix-list radius-address {
 /* Your radius server */
 }
 prefix-list tacacs-address {
 /* The ip of the tacacs */
 }
 prefix-list isis-address {
 /* The ranges you are running ISIS on */
 }
 prefix-list management-address {
 /* The IP you want to allow management to */
 }
 prefix-list static-to-isis {
 /* Range to redistribute from static to ISIS (so they diseapar if the link goes down) */
 }
 policy-statement blackhole {
 term rewrite-next-hop {
 from {
 protocol bgp;
 community blackhole-here;
 }
 then {
 community add no-export;
 next-hop 127.0.0.2;
 accept;
 }
 }
 }
 policy-statement damping {
 term 1 {
 from {
 prefix-list root-servers;
 }
 then {
 damping damp-none;
 next policy;
 }
 }
 term 2 {
 from {
 route-filter 0.0.0.0/0 upto /21 damping damp-short;
 route-filter 0.0.0.0/0 upto /23 damping damp-medium;
 route-filter 0.0.0.0/0 orlonger damping damp-long;
 }
 then next policy;
 }
 }
 policy-statement deny-all {
 then reject;
 }
 policy-statement export-customer {
 term remove {
 from {
 protocol bgp;
 community withdraw-customer;
 }
 then reject;
 }
 term prepend-one-time {
 from {
 protocol bgp;
 community prepend1-customer;
 }
 then as-path-prepend 54321;
 }
 term prepend-two-times {
 from {
 protocol bgp;
 community prepend2-customer;
 }
 then as-path-prepend "54321 54321";
 }
 term prepend-four-times {
 from {
 protocol bgp;
 community prepend4-customer;
 }
 then as-path-prepend "54321 54321 54321 54321";
 }
 }
 policy-statement export-ibgp {
 term remove-community {
 from {
 protocol bgp;
 community withdraw-ibgp;
 }
 then reject;
 }
 }
 policy-statement export-linx {
 term remove {
 from {
 protocol bgp;
 community withdraw-linx;
 }
 then reject;
 }
 term prepend-one-time {
 from {
 protocol bgp;
 community prepend1-linx;
 }
 then as-path-prepend 54321;
 }
 term prepend-two-times {
 from {
 protocol bgp;
 community prepend2-linx;
 }
 then as-path-prepend "54321 54321";
 }
 term prepend-four-times {
 from {
 protocol bgp;
 community prepend4-linx;
 }
 then as-path-prepend "54321 54321 54321 54321";
 }
 }
 policy-statement export-peering {
 term remove-peering {
 from {
 protocol bgp;
 community route-peering;
 }
 then reject;
 }
 term remove-transit {
 from {
 protocol bgp;
 community route-transit;
 }
 then reject;
 }
 term remove-community {
 from {
 protocol bgp;
 community withdraw-peering;
 }
 then reject;
 }
 term prepend-one-time {
 from {
 protocol bgp;
 community prepend1-peering;
 }
 then as-path-prepend 54321;
 }
 term prepend-two-times {
 from {
 protocol bgp;
 community prepend2-peering;
 }
 then as-path-prepend "54321 54321";
 }
 term prepend-four-times {
 from {
 protocol bgp;
 community prepend4-peering;
 }
 then as-path-prepend "54321 54321 54321 54321";
 }
 }
 policy-statement export-transit {
 term remove-peering {
 from {
 protocol bgp;
 community route-peering;
 }
 then reject;
 }
 term remove-transit {
 from {
 protocol bgp;
 community route-transit;
 }
 then reject;
 }
 term remove-community {
 from {
 protocol bgp;
 community withdraw-transit;
 }
 then reject;
 }
 term prepend-one-time {
 from {
 protocol bgp;
 community prepend1-transit;
 }
 then as-path-prepend 54321;
 }
 term prepend-two-times {
 from {
 protocol bgp;
 community prepend2-transit;
 }
 then as-path-prepend "54321 54321";
 }
 term prepend-four-times {
 from {
 protocol bgp;
 community prepend4-transit;
 }
 then as-path-prepend "54321 54321 54321 54321";
 }
 }
 policy-statement export-transit1 {
 term remove {
 from {
 protocol bgp;
 community withdraw-transit1;
 }
 then reject;
 }
 term prepend-one-time {
 from {
 protocol bgp;
 community prepend1-transit1;
 }
 then as-path-prepend 54321;
 }
 term prepend-two-times {
 from {
 protocol bgp;
 community prepend2-transit1;
 }
 then as-path-prepend "54321 54321";
 }
 term prepend-four-times {
 from {
 protocol bgp;
 community prepend4-transit1;
 }
 then as-path-prepend "54321 54321 54321 54321";
 }
 }
 /* Load balance packet through all possible routes */
 policy-statement load-balancing {
 then {
 load-balance per-packet;
 }
 }
 policy-statement local-preference-customer {
 term more {
 from {
 protocol bgp;
 community local_preference_12;
 }
 then {
 local-preference 300;
 }
 }
 term normal {
 from {
 protocol bgp;
 community local_preference_11;
 }
 then {
 local-preference 275;
 }
 }
 term less {
 from {
 protocol bgp;
 community local_preference_10;
 }
 then {
 local-preference 250;
 }
 }
 }
 policy-statement local-preference-peer {
 term default {
 from protocol bgp;
 then {
 local-preference 175;
 }
 }
 term more {
 from {
 protocol bgp;
 community local_preference_08;
 }
 then {
 local-preference 200;
 }
 }
 term normal {
 from {
 protocol bgp;
 community local_preference_07;
 }
 then {
 local-preference 175;
 }
 }
 term less {
 from {
 protocol bgp;
 community local_preference_06;
 }
 then {
 local-preference 150;
 }
 }
 }
 policy-statement local-preference-transit {
 term default {
 from protocol bgp;
 then {
 local-preference 75;
 }
 }
 }
 policy-statement next-hop-self {
 then {
 next-hop self;
 }
 }
 policy-statement no-bogons {
 term default-route {
 from {
 route-filter 0.0.0.0/0 businessct;
 }
 then reject;
 }
 term reserved {
 from {
 route-filter 10.0.0.0/8 orlonger;
 route-filter 172.16.0.0/12 orlonger;
 route-filter 192.168.0.0/16 orlonger;
 route-filter 169.254.0.0/16 orlonger;
 route-filter 192.0.2.0/24 orlonger;
 route-filter 240.0.0.0/4 orlonger;
 route-filter 192.42.172.0/24 orlonger;
 route-filter 198.18.0.0/15 orlonger;
 route-filter 127.0.0.0/8 orlonger;
 }
 then reject;
 }
 term multicast {
 from {
 route-filter 224.0.0.0/4 orlonger;
 }
 then reject;
 }
 term too-short {
 from {
 route-filter 0.0.0.0/0 prefix-length-range /0-/5;
 }
 then reject;
 }
 }
 policy-statement no-community-export {
 then {
 community delete blackhole-everywhere;
 community delete originate;
 community delete originate-customer;
 community delete internal;
 }
 }
 policy-statement no-community-import {
 then {
 community delete originate;
 community delete originate-customer;
 community delete route-customer;
 community delete internal;
 }
 }
 policy-statement no-export {
 then {
 community add no-export;
 }
 }
 policy-statement no-ix {
 from {
 /* Enlix */
 route-filter 193.189.130.0/24 orlonger reject;
 /* LINX */
 route-filter 195.66.224.0/22 orlonger reject;
 }
 then reject;
 }
 policy-statement no-leak {
 term remove-path {
 from {
 protocol bgp;
 as-path [ leaked-quest leaked-verizon-na leaked-verizon-eu leaked-verizon-ap leaked-sprint leaked-telia leaked-atdn leaked-tiscali leaked-deutsche-telekom leaked-level3 leaked-savvis leaked-france-telecom leaked-telecom-italia leaked-att leaked-ntt leaked-global-crossing leaked-vsnl leaked-cogent ];
 }
 then reject;
 }
 }
 policy-statement no-small-prefixes {
 from {
 route-filter 0.0.0.0/0 prefix-length-range /25-/32 reject;
 }
 then reject;
 }
 policy-statement no-transit {
 term remove-path {
 from {
 protocol bgp;
 as-path [ transit1-routes ];
 }
 then reject;
 }
 }
 policy-statement originate-community {
 from community originate;
 then {
 next-hop self;
 accept;
 }
 }
 policy-statement originate-customer {
 from community originate-customer;
 then {
 next-hop self;
 accept;
 }
 }
 policy-statement originate-default {
 from {
 route-filter 0.0.0.0/0 businessct;
 }
 then accept;
 }
 policy-statement static-to-isis {
 from {
 protocol static;
 prefix-list static-to-isis;
 }
 to protocol isis;
 then accept;
 }
 policy-statement tag-customer {
 then {
 community add route-customer;
 }
 }
 policy-statement tag-linx {
 then {
 community add route-linx;
 }
 }
 policy-statement tag-peering {
 then {
 community add route-peering;
 }
 }
 policy-statement tag-transit {
 then {
 community add route-transit;
 }
 }
 policy-statement tag-transit1 {
 then {
 community add route-transit1;
 }
 }
 community blackhole-customer members 65100:65004;
 community blackhole-everywhere members [ 65100:65001 65100:65002 65100:65003 65100:65004 ];
 community blackhole-here members [ 65100:65001 65100:65002 65100:65003 65100:65004 ];
 community blackhole-ibgp members 65100:65001;
 community blackhole-peering members 65100:65002;
 community blackhole-transit members 65100:65003;
 /* Cymru communities */
 community internal members [ 65000:* 65001:* 65002:* 65003:* 65004:* 65100:* ];
 community local_preference_01 members 65005:65201;
 community local_preference_02 members 65005:65202;
 community local_preference_03 members 65005:65203;
 community local_preference_04 members 65005:65204;
 community local_preference_05 members 65005:65205;
 community local_preference_06 members 65005:65206;
 community local_preference_07 members 65005:65207;
 community local_preference_08 members 65005:65208;
 community local_preference_09 members 65005:65209;
 community local_preference_10 members 65005:65210;
 community local_preference_11 members 65005:65211;
 community local_preference_12 members 65005:65212;
 community local_preference_13 members 65005:65213;
 community no-export members no-export;
 community originate members 54321:54321;
 community originate-customer members 54321:0;
 community prepend1-customer members 65001:65004;
 community prepend1-linx members 65001:5459;
 community prepend1-peering members 65001:65002;
 community prepend1-transit members 65001:65003;
 community prepend1-transit1 members 65001:1234;
 community prepend2-customer members 65002:65004;
 community prepend2-linx members 65002:5459;
 community prepend2-peering members 65002:65002;
 community prepend2-transit members 65002:65003;
 community prepend2-transit1 members 65002:1234;
 community prepend4-customer members 65004:65004;
 community prepend4-linx members 65004:5459;
 community prepend4-peering members 65004:65002;
 community prepend4-transit members 65004:65003;
 community prepend4-transit1 members 65004:1234;
 community route-customer members 54321:65004;
 community route-ibgp members 54321:65001;
 community route-linx members 54321:5459;
 community route-peering members 54321:65002;
 community route-transit members 54321:65003;
 community route-transit1 members 54321:1234;
 community routes-dsl members 54321:65101;
 community routes-mpls members 54321:65102;
 community routes-transit1 members 54321:1234;
 community withdraw-customer members 65000:65004;
 community withdraw-everywhere members [ 65000:65001 65000:65002 65000:65003 65000:65004 ];
 community withdraw-ibgp members 65000:65001;
 community withdraw-linx members 65000:5459;
 community withdraw-peering members 65000:65002;
 community withdraw-transit members 65000:65003;
 community withdraw-transit1 members 65000:1234;
 as-path private-asn-range 64512-65535;
 as-path leaked-quest ".{1,}209.*";
 as-path leaked-verizon-na ".{1,}701.*";
 as-path leaked-verizon-eu ".{1,}702.*";
 as-path leaked-verizon-ap ".{1,}703.*";
 as-path leaked-sprint ".{1,}1239.*";
 as-path leaked-telia ".{1,}1299.*";
 as-path leaked-atdn ".{1,}1668.*";
 as-path leaked-tiscali ".{1,}3257.*";
 as-path leaked-deutsche-telekom ".{1,}3320.*";
 as-path leaked-level3 ".{1,}3356.*";
 as-path leaked-savvis ".{1,}3561.*";
 as-path leaked-france-telecom ".{1,}5511.*";
 as-path leaked-telecom-italia ".{1,}6762.*";
 as-path leaked-att ".{1,}7018.*";
 as-path leaked-ntt ".{1,}1914.*";
 as-path leaked-global-crossing ".{1,}3549.*";
 as-path leaked-vsnl ".{1,}6453.*";
 as-path leaked-cogent ".{1,}174.*";
 as-path transit1-routes 1234.*;
 /* Min: 30 min, Max: 60 min, dampen at 3 flaps */
 damping damp-long {
 half-life 30;
 reuse 1640;
 suppress 6000;
 max-suppress 60;
 }
 /* Min: 15 min, Max: 45 min, dampen at 3 flaps */
 damping damp-medium {
 half-life 15;
 reuse 1500;
 suppress 6000;
 max-suppress 45;
 }
 /* Min: 10 min, Max: 30 min, dampen at 3 flaps */
 damping damp-short {
 half-life 10;
 reuse 3000;
 suppress 6000;
 max-suppress 30;
 }
 /* Do not dampen */
 damping damp-none {
 disable;
 }
}

firewall {
 filter external-outgoing {
 term valid-outgoing-traffic { }
 term log-spoofing { }
 }
 filter flood-detect {
 term tcp-syn-count { }
 term tcp-rst-count { }
 term tcp-fin-count { }
 term tcp-allow { }
 term udp-allow { }
 }
 filter protect-bgp {
 term bgp-connection-limit { }
 term bgp-allow { }
 term default-deny { }
 }
 filter protect-management {
 term icmp-limit { }
 term trace-route-limit { }
 term ssh-connection-limit { }
 term ssh-limit { }
 term dns-limit { }
 term ntp-limit { }
 term snmp-limit { }
 term auth-limit { }
 term telnet-limit { }
 term default-deny { }
 }
 filter protect-icmp {
 term icmp-allow { }
 term default-deny { }
 }
 filter protect-isis {
 term isis-connection-limit { }
 term isis-allow { }
 term default-deny { }
 }
 filter external-incoming-customer {
 term transfer-allow { }
 term originate-deny { }
 term peer-deny { }
 term transit-deny { }
 term rfc1918-deny { }
 term manangement-allow { }
 term infrastructure-icmp-allow { }
 term infrastructure-deny { }
 term icmp-limit { }
 term multicast-limit { }
 term default-allow { }
 }
 filter external-incoming-transit {
 term transfer-allow { }
 term originate-deny { }
 term peer-deny { }
 term customer-deny { }
 term free-transit-deny { }
 term rfc1918-deny { }
 term manangement-allow { }
 term infrastructure-icmp-allow { }
 term infrastructure-deny { }
 term icmp-limit { }
 term multicast-limit { }
 term default-allow { }
 }
 filter external-incoming-peer {
 term transfer-allow { }
 term originate-deny { }
 term customer-deny { }
 term transit-deny { }
 term free-transit-deny { }
 term rfc1918-deny { }
 term manangement-allow { }
 term infrastructure-icmp-allow { }
 term infrastructure-deny { }
 term icmp-limit { }
 term multicast-limit { }
 term default-allow { }
 }
 filter sample-netflow { }
 filter ddos-protect { }
}

Vmailmgr Proxy

Sun, 21 Oct 2007 12:16:00 GMT

In order to horizontally scale our mail cluster, I have developed a vmailmgr proxy using twisted.

The code has been running without issue for a few months in our network, and I am not aware of any issues (I would still recommend that you do not let this proxy unfirewalled).

You can download the code here

As this is an adaptation of our code which removes all dependencies on our own internal library, it is not as polished as it should be, but should still "just work".

It is possible to force a reload of the configuration file sending an HUP signal to the server.

Ideally the twistd daemon should be supervised.

I noticed that the smtp server used for email notification is hardcoded in one of library, do not forget to change it if you want the feature to work.

IRRDT

Sat, 15 Sep 2007 17:31:00 GMT

IRR dummy tools

First get the good stuff

IRRPT provides a great way to gather and track prefixes announced by ebgp speakers but does not provide anything to help you to keep the configuration files and your network in sync. This is what this tools do.

IRRDT is written in Python but will will parse the PHP configuration of IIRPT (as long you are not using some weird PHP syntax).
It will as well require some extra configuration option in the file.

Warning ..

This code is a work in progress, this site was only announced at LINX 57 and did not see much improvement since. It have a few known rought corners.

How to use it

First get it here
(the previous version is here)

I will write some good docs but for the moment you mostly will have to figure how it works yourself with the examples below. Just make sure you save your config to a local folder using juniper archival feature

[edit system archival] 
configuration {
    transfer-on-commit;
        archive-sites {
            "ftp://user:password@server/router-name/";
        }
    }
}

Each of your neighbour should have a peer-as set for the parser to work and a comment of the form:

"AS-ACCEPTED | Peer name | noc@peer.co.uk | AS-SENT"

AS-SENT (or ASN), is optional as otherwise taken from the tools command line.

IRRDT is looking for the following configuration options

$cfg['global']['asn'] = "12345";
$cfg['global']['as-macro'] = "AS-YOURSELF";
$cfg['global']['parse']	= "juniper";

$cfg['paths']['juniper'] = "/location/of/your/ftpd/juniper/files";
$cfg['juniper']['peer'] = "export-peer";
$cfg['juniper']['transit'] = "export-transit";
$cfg['juniper']['customer'] = "export-customer";
$cfg['irrdb']['export'] = "peer customer transit";

$cfg['ripe']['transit'] = "0";
$cfg['ripe']['peer'] = "50";
$cfg['ripe']['customer'] = "";

you can create a file called "ripe.secret" in the conf/ directory to get you mail to ripe accepted (if you have a

The names should be self-explanatory. I only wrote a Juniper configuration parser. The type of connection is detected inspecting the export and import statement for special policy-options. The code is expecting the peers and transit to be in different groups (which names are used in the ripe output).

An example would be like follows:

group transit {
    type external;
    local-preference 75;
    remove-private;
    neighbor 195.219.195.45 {
        inactive: traceoptions {
            file bgp-vsnl size 1m files 5;
        }
        description "ANY | Teleglobe / VSNL | email@vsnlinternational.com |";
        local-address 195.219.195.46;
        import [ no-ix no-bogons no-small-prefixes tag-transit tag-vsnl damping local-preference-transit no-community-import ];
        export [ originate-community originate-customer no-transit no-small-prefixes export-transit export-vsnl no-community-export next-hop-self ];
        peer-as 6453;
    }
}

Usage example

There is two way to use the tools :

use the ripe.py file which is taking all its info from the configuration files
look at the ripe.sh file which allow you to use only part of the code to better fit your needs

For the ripe.py, generate your irrdb.conf, which allow you to fetch the peers prefixes

aggregate

The aggregate program recommended by RAS seems to have a serious O(exp n) problem - we speak in term of 10's of minutes to get a large sets of IP aggregated.
I have write a python program aggregate.py performing the same aggregation (and is still suboptimal) but perform the same work in a few seconds.

Generating the prefix

The tools to generate the prefix are currently broken ... come back later or try (the syntax and command location may have changed):

[thomas@linx-meeting-57 app]$ ./import/juniper.py ../../ftpd/ | grep router-name | xargs cat - | parse/juniper.py 30740 AS-EXA -p export-peer -c export-customer -t export-transit | ./export/asn.py | xargs ./export/prefix.py