Hello!

I have an ER-X with actual Firmware which connects to an OpenVPN Server!

On the ER-X side i try to get the DHPC-Relay working.

I can see that the dhcp packets sent trough the OpenVPN connection to the server and the server sent the answer back. But the ER-X does not forward it to the correct port!

I read some posts and there was the hint to add also the vtun0 interface but this does not work.

I can see in /var/log/messages an error text that vtun0 could not bind!

What can i do to get it working?

This is the Unifi forum (USG). You need the EdgeRouter forum.

https://community.ubnt.com/t5/EdgeRouter/bd-p/EdgeMAX

You need 4 NAT rules like 5011, one for each LAN, translating internal subnet into a single IP.

Get rid of the masquerade (or make it the last rule), now it catches all traffic.

"no internet" is a lousy problem description.

What doesn't work?

get dhcp address

ping 8.8.8.8

ping by name

....

ER has build in sniffer

sudo tcpdump -i eth0 -n -v

will show traffic on WAN,

Hi, I have two upstream ISP providers, and I'd like to run DHCP6-PD with both of them at the same time for failover et al.

(yes, I know about the NPTv6 issues, don't siderail this conversation with that)

There is a bug in the way dhcp6c configurations are written out. dhcp6c is intended to run only ONCE, not once per interface.

The Vyatta OS is writing out two separate configuration files, and attempting to run dhcpc6 twice, which is incorrect.  The second instance of dhcpc6 will always fail because bother instances are trying to bind to a HARD CODED control port in the image.

The proper way to run dhcp6c is once, with a unified configuration file, that might look like this:

# This file was auto-generated by /opt/vyatta/sbin/dhcpv6-pd-client.pl
# configuration sub-system.  Do not edit it.

interface eth0 {
	send ia-na 0;
	request domain-name-servers, domain-name;
	send rapid-commit;
	send ia-pd 0;
	script "/opt/vyatta/sbin/ubnt-dhcp6c-script";
};

interface eth1 {
	send ia-na 0;
	request domain-name-servers, domain-name;
	send rapid-commit;
	send ia-pd 1;
	script "/opt/vyatta/sbin/ubnt-dhcp6c-script";
};

id-assoc na 0 {};

id-assoc pd 0 {
	prefix ::/56 infinity;
	prefix-interface eth2 {
		sla-id 1;
		sla-len 8;
		ifid 1;
	};
	prefix-interface eth2.3 {
		sla-id 2;
		sla-len 8;
		ifid 1;
	};
	prefix-interface eth3 {
		sla-id 3;
		sla-len 8;
		ifid 1;
	};
};

id-assoc pd 1 {
	prefix ::/60 infinity;
	prefix-interface eth2 {
		sla-id 1;
		sla-len 4;
		ifid 1;
	};
	prefix-interface eth2.3 {
		sla-id 2;
		sla-len 4;
		ifid 1;
	};
	prefix-interface eth3 {
		sla-id 3;
		sla-len 4;
		ifid 1;
	};
};

Please take note that there is only 1 ia-notifcation clause, and pd 1 must be used for the second interface.

0x2002  in dotted decimal is 32.2

0x2002/16  in dotted decimal is 32.2.x.x........x   (x means: bit content = don't care)

0x2002/17 means 32.2 , and the next bit must be zero.  this matches 32.2.[0,1.....127].x.x.x.....x

So, bitwise this is logical.  

But applying bitwise ipv6 filter on ipv4 routes should be considered a bug

Following up, this is with the latest code (2.10), and if you configure things as one would have expected from the documentation, the second instance fails with:

sudo /usr/sbin/dhcp6c -c /var/run/dhcp6c-eth1-pd.conf -p /var/run/dhcp6c-eth1-pd.pid -df eth1

May/31/2019 12:59:19: dhcp6_ctl_init: bind(control sock): Address already in use
May/31/2019 12:59:19: client6_init: failed to initialize control channel

Perusing the source code of dhcp6c shows that things are hard-coded. Further perusal of the documentation shows that dhcp6c is intended to be run only once per box.

The configuration snippet that generates this failure looks like this:

ethernet eth0 {
     address dhcp
     description WAN1/Primary
     dhcpv6-pd {
         pd 0 {
             interface eth2 {
                 host-address ::1
                 prefix-id :1
                 service dhcpv6-stateless
             }
             interface eth2.3 {
                 host-address ::1
                 prefix-id :2
                 service dhcpv6-stateless
             }
             interface eth3 {
                 host-address ::1
                 prefix-id :3
                 service dhcpv6-stateless
             }
             prefix-length /56
         }
         rapid-commit enable
     }
     duplex auto
     firewall {
         in {
             ipv6-name WANv6_IN
             name WAN_IN
         }
         local {
             ipv6-name WANv6_LOCAL
             name WAN_LOCAL
         }
     }
     speed auto
 }
 ethernet eth1 {
     address dhcp
     description WAN2/Backup
     dhcpv6-pd {
         pd 0 {
             interface eth2 {
                 host-address ::1
                 prefix-id :1
                 service dhcpv6-stateless
             }
             interface eth2.3 {
                 host-address ::1
                 prefix-id :2
                 service dhcpv6-stateless
             }
             interface eth3 {
                 host-address ::1
                 prefix-id :3
                 service dhcpv6-stateless
             }
             prefix-length /60
         }
         rapid-commit enable
     }
     duplex auto
     firewall {
         in {
             ipv6-name WANv6_IN
             name WAN_IN
         }
         local {
             ipv6-name WANv6_LOCAL
             name WAN_LOCAL
         }
     }
     speed auto
 }

Well, let's see:

1. The default-router in your DHCP pools eth1-* is wrong.
2. NAT rule 5010 (masquerade) must be after the SNAT rules as I mentioned in my last post.
3. NAT rule 5011 only matches the router interface -- you probably want the source to be the whole subnet.

Fix that and try again.

My 1st thought:

This isn't a DDoS attack.  You have some SIP phone device, and mapped port 5060 to it (which isn't necessary)

Now the outside world portscanners detect your open port, and are brute forcing the login

Thx i will try that.

how did you apply this prefix-list?  Did you specify "address-family ipv6-unicast prefix-list" or just "prefix-list" in your bgp peer configuration?

I've got WAN1 interface ethernet 0 disabled:

set interface ethernet eth0 disabled

commit

dhcp6c is still started for this interface.

Additonal bug:

There is no

set interface ethernet eth0 dhcpv6-pd disable

or

set interface ethernet eth0 dhcpv6-pd pd 0 disable

commands in the OS, so there is no way to temporarily remove dhcpv6-pd from an interface.

If I understand right, the failure here is that I was applying a stateful firewall rule to a stateless situation.  Because the icmp reply coming in was neither, New, Established, or Related, it was, by default, Invalid.  A perfectly valid packet that was invalid.

you need to downgrade from 2.0.1 and 2.0.2 to 10.9 branch before upgrading due to  firmware with HWNAT enabled.   Rule of thumb always downgrade to stable branch before upgrading to latest stable..  My assumption is that UBNT QA tests from latest stable to latest stable, not from beta to stable.

Just my 2 cents worth.

  Is TCP FAST OPEN enabled in this kernel revision, this would account for the 5-10% decrease in speed noted in the changelog.

I'm having trouble getting a static route to function, the same route works on my EdgeRouter 4 but doesnt fix the problem on my EdgeRouter 6P.

My EdgeRouter 6P has the IP 50.228.117.92, it is via Static from my RV042 configured in Bridge Mode

Looks like this:

Comcast Ethernet
Gateway/30: 50.228.117.49
IP/30: 50.228.117.50 - RV042 Router
DNS: 75.75.75.75
DNS: 8.8.8.8

IP Block:

50.228.117.64/27

Usable: 50.228.117.65-50.228.117.94

Currently most of the IPs are being One-To-One NAT'd on the RV042 to my Server

50.228.117.65 - 50.228.117.91

EdgeRouter 4

50.228.117.92/24

Local

192.168.6.0/24

EdgeRouter 6P

50.228.117.93

Local

192.168.1.0/24

I was having trouble accessing 50.228.117.89 from the 192.168.6 local subnet, I added this static route and the problem was fixed:

50.228.117.64/27 Next Hop: 50.228.117.49

I tried adding that static route to the EdgeRouter 6P and it just doesn't work.

I'm thinking I might have misconfigured the IP situation and thats blocking local hosts from getting to IPs in my block. I've attached my config, please let me know if you see what I've done wrong. Thanks in advance!

CONFIG

----------------

firewall {
    all-ping enable
    broadcast-ping disable
    group {
        network-group PRIVATE_NETS {
            network 192.168.0.0/16
            network 172.16.0.0/12
            network 10.0.0.0/8
        }
    }
    ipv6-receive-redirects disable
    ipv6-src-route disable
    ip-src-route disable
    log-martians disable
    modify balance {
        rule 10 {
            action modify
            description "do NOT load balance lan to lan"
            destination {
                group {
                    network-group PRIVATE_NETS
                }
            }
            modify {
                table main
            }
        }
        rule 20 {
            action modify
            description "do NOT load balance destination public address"
            destination {
                group {
                    address-group ADDRv4_eth0
                }
            }
            modify {
                table main
            }
        }
        rule 30 {
            action modify
            description "do NOT load balance destination public address"
            destination {
                group {
                    address-group ADDRv4_eth1
                }
            }
            modify {
                table main
            }
        }
        rule 110 {
            action modify
            modify {
                lb-group G
            }
        }
    }
    name WAN_IN {
        default-action drop
        description "WAN to internal"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
    }
    name WAN_LOCAL {
        default-action drop
        description "WAN to router"
        rule 10 {
            action accept
            description "Allow established/related"
            state {
                established enable
                related enable
            }
        }
        rule 20 {
            action drop
            description "Drop invalid state"
            state {
                invalid enable
            }
        }
        rule 21 {
            action accept
            description "Remote Management"
            destination {
                port 80,443,22
            }
            log disable
            protocol tcp
        }
        rule 30 {
            action accept
            description openvpn
            destination {
                port 1194
            }
            protocol udp
        }
    }
    receive-redirects disable
    send-redirects enable
    source-validation disable
    syn-cookies enable
}
interfaces {
    ethernet eth0 {
        address 50.228.117.93/24
        description WAN
        duplex auto
        firewall {
            in {
                name WAN_IN
            }
            local {
                name WAN_LOCAL
            }
        }
        poe {
            output off
        }
        speed auto
    }
    ethernet eth1 {
        address 173.12.37.88/24
        description "WAN 2"
        disable
        duplex auto
        firewall {
            in {
                name WAN_IN
            }
            local {
                name WAN_LOCAL
            }
        }
        speed auto
    }
    ethernet eth2 {
        duplex auto
        speed auto
    }
    ethernet eth3 {
        address 192.168.1.1/24
        description Local
        duplex auto
        firewall {
            in {
                modify balance
            }
        }
        speed auto
    }
    ethernet eth4 {
        duplex auto
        speed auto
    }
    ethernet eth5 {
        duplex auto
        speed auto
    }
    loopback lo {
    }
    openvpn vtun0 {
        mode server
        server {
            name-server 192.168.1.1
            push-route 192.168.1.0/24
            subnet 172.16.1.0/24
        }
        tls {
            ca-cert-file /config/auth/cacert.pem
            cert-file /config/auth/server.pem
            dh-file /config/auth/dh.pem
            key-file /config/auth/server.key
        }
    }
}
load-balance {
    group G {
        interface eth0 {
        }
        interface eth1 {
            failover-only
        }
        lb-local enable
        lb-local-metric-change disable
    }
}
port-forward {
    auto-firewall enable
    hairpin-nat enable
    lan-interface eth3
    rule 1 {
        description PBX1
        forward-to {
            address 192.168.1.40
            port 8000-20000
        }
        original-port 8000-20000
        protocol udp
    }
    rule 2 {
        description PBX2
        forward-to {
            address 192.168.1.40
            port 5000-6500
        }
        original-port 5000-6500
        protocol tcp_udp
    }
    rule 3 {
        description PBX
        forward-to {
            address 192.168.1.40
            port 123
        }
        original-port 123
        protocol tcp
    }
    wan-interface eth0
}
protocols {
    static {
        route 0.0.0.0/0 {
            next-hop 50.228.117.49 {
            }
            next-hop 173.12.37.94 {
                disable
            }
        }
        route 50.228.117.64/27 {
            next-hop 50.228.117.49 {
            }
        }
    }
}
service {
    dhcp-server {
        disabled false
        hostfile-update disable
        shared-network-name LAN {
            authoritative enable
            subnet 192.168.1.0/24 {
                default-router 192.168.1.1
                dns-server 192.168.1.1
                dns-server 8.8.8.8
                lease 86400
                start 192.168.1.100 {
                    stop 192.168.1.199
                }
                static-mapping GrandstreamBase {
                    ip-address 192.168.1.101
                    mac-address 00:0b:82:9d:56:3a
                }
                static-mapping GrandstreamRepeater {
                    ip-address 192.168.1.102
                    mac-address 00:0b:82:d7:b6:ec
                }
            }
        }
        static-arp disable
        use-dnsmasq disable
    }
    dns {
        forwarding {
            cache-size 150
            listen-on eth3
            listen-on vtun0
        }
    }
    gui {
        http-port 80
        https-port 443
        older-ciphers enable
    }
    nat {
        rule 5000 {
            description "masquerade for WAN"
            outbound-interface eth0
            type masquerade
        }
        rule 5002 {
            description "masquerade for WAN 2"
            outbound-interface eth1
            type masquerade
        }
    }
    ssh {
        port 22
        protocol-version v2
    }
    unms {
        disable
    }
}
system {
    conntrack {
        expect-table-size 4096
        hash-size 4096
        table-size 32768
        tcp {
            half-open-connections 512
            loose enable
            max-retrans 3
        }
    }
    host-name Shops1812
    login {
        user ryan {
            authentication {
                encrypted-password $5$E6PEdsboUJyaAjv/$wkTYceGM994larplgjrIS1CLbOOsX2CU6qOzaZ/mzY0
                plaintext-password ""
            }
            level operator
        }
        user sdm {
            authentication {
                encrypted-password $5$wd6PSzOQaGJ5ITUU$P6QKfbIQGKDX7rmPjMQvwNFX2dLqfy7irYkgfVGBOh1
                plaintext-password ""
            }
            level admin
        }
        user shops {
            authentication {
                encrypted-password $6$Cixb7PDOC$AVI2CAMAQq.IuMMcEjUiOoVHLMYV0638soOUo2ZOtZslCq/JjksmalbY/28YhjTt0bWIUt/e6Bdb7fC9GoAk6/
            }
            level admin
        }
    }
    name-server 75.75.75.75
    name-server 8.8.8.8
    ntp {
        server 0.ubnt.pool.ntp.org {
        }
        server 1.ubnt.pool.ntp.org {
        }
        server 2.ubnt.pool.ntp.org {
        }
        server 3.ubnt.pool.ntp.org {
        }
    }
    offload {
        hwnat disable
        ipsec enable
        ipv4 {
            bonding enable
            forwarding enable
            gre enable
            pppoe enable
            vlan enable
        }
    }
    syslog {
        global {
            facility all {
                level notice
            }
            facility protocols {
                level debug
            }
        }
    }
    time-zone America/New_York
}
vpn {
    ipsec {
        allow-access-to-local-interface disable
        auto-firewall-nat-exclude enable
        esp-group FOO0 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes256
                hash sha1
            }
        }
        esp-group FOO1 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes256
                hash sha1
            }
        }
        esp-group FOO2 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes256
                hash sha1
            }
        }
        esp-group FOO3 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes256
                hash sha1
            }
        }
        esp-group FOO4 {
            compression disable
            lifetime 3600
            mode tunnel
            pfs enable
            proposal 1 {
                encryption aes128
                hash sha1
            }
        }
        ike-group FOO0 {
            ikev2-reauth no
            key-exchange ikev1
            lifetime 28800
            proposal 1 {
                dh-group 5
                encryption aes256
                hash sha1
            }
        }
        ike-group FOO1 {
            ikev2-reauth no
            key-exchange ikev1
            lifetime 28800
            proposal 1 {
                dh-group 5
                encryption aes256
                hash sha1
            }
        }
        ike-group FOO2 {
            ikev2-reauth no
            key-exchange ikev1
            lifetime 28800
            proposal 1 {
                dh-group 2
                encryption aes256
                hash sha1
            }
        }
        ike-group FOO3 {
            ikev2-reauth no
            key-exchange ikev1
            lifetime 28800
            proposal 1 {
                dh-group 5
                encryption aes256
                hash sha1
            }
        }
        ike-group FOO4 {
            ikev2-reauth no
            key-exchange ikev1
            lifetime 28800
            proposal 1 {
                dh-group 14
                encryption aes128
                hash sha1
            }
        }
        site-to-site {
            peer ajmshouse.dvrdns.org {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret 1812wep1812
                }
                connection-type initiate
                description Adam
                ike-group FOO0
                ikev2-reauth inherit
                local-address any
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    esp-group FOO0
                    local {
                        prefix 192.168.1.0/24
                    }
                    remote {
                        prefix 192.168.2.0/24
                    }
                }
            }
            peer kristen.dvrdns.org {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret 1812wep1812
                }
                connection-type initiate
                description Kristen
                ike-group FOO1
                ikev2-reauth inherit
                local-address any
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    esp-group FOO1
                    local {
                        prefix 192.168.1.0/24
                    }
                    remote {
                        prefix 192.168.4.0/24
                    }
                }
            }
            peer shopsin.dvrdns.org {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret 1812wep1812
                }
                connection-type initiate
                description "Shops IN"
                ike-group FOO2
                ikev2-reauth inherit
                local-address any
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    esp-group FOO2
                    local {
                        prefix 192.168.1.0/24
                    }
                    remote {
                        prefix 192.168.6.0/24
                    }
                }
            }
            peer vernice.dvrdns.org {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret 1812wep1812
                }
                connection-type initiate
                description Vernice
                ike-group FOO3
                ikev2-reauth inherit
                local-address any
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    esp-group FOO3
                    local {
                        prefix 192.168.1.0/24
                    }
                    remote {
                        prefix 192.168.5.0/24
                    }
                }
            }
            peer wimplo81.dvrdns.org {
                authentication {
                    mode pre-shared-secret
                    pre-shared-secret 1812wep1812
                }
                connection-type initiate
                description Spencer
                ike-group FOO4
                ikev2-reauth inherit
                local-address any
                tunnel 1 {
                    allow-nat-networks disable
                    allow-public-networks disable
                    esp-group FOO4
                    local {
                        prefix 192.168.1.0/24
                    }
                    remote {
                        prefix 192.168.3.0/24
                    }
                }
            }
        }
    }
    pptp {
        remote-access {
            authentication {
                local-users {
                    username ryan {
                        password hH6kmsn#B69u
                    }
                    username sdm {
                        password 4o$6PW8r!D2W
                    }
                }
                mode local
            }
            client-ip-pool {
                start 192.168.1.200
                stop 192.168.1.204
            }
            dns-servers {
                server-1 192.168.1.1
                server-2 8.8.8.8
            }
            mtu 1492
            outside-address 50.228.117.93
        }
    }
}


/* Warning: Do not remove the following line. */
/* === vyatta-config-version: "config-management@1:conntrack@1:cron@1:dhcp-relay@1:dhcp-server@4:firewall@5:ipsec@5:nat@3:qos@1:quagga@2:suspend@1:system@4:ubnt-pptp@1:ubnt-udapi-server@1:ubnt-unms@1:ubnt-util@1:vrrp@1:webgui@1:webproxy@1:zone-policy@1" === */
/* Release version: v2.0.1.5174690.190312.1614 */

Is 2.0.1 'beta'? It's listed on the normal download page along with 1.0.9 and 2.0.3.

Indeed you're right.

In triangular routing, routers see one direction of the packet flow

Valid reply packets are considered invalid by the router, as it hasn't seen the request passing by.

Ping and TCP won't work,  UDP will.

Hi  ,

The 24v PoE output of an ER-X-SFP is interrupted during a reboot, causing a power cycle to any powered/connected device.

Best regards,

What tripped me up was that I didn't realize the icmp reply packet wasn't classified as a "New" packet to the WAN 1 router.