We have 18 remote sites connecting back over T1, need to configure MetroE in similar fashion 1

[findmefast]

Referincing: thread557-1474090

burtsbees said, "My FastEthernet 0/1 interface needs to be configured to allow our 18 remote locations to connect back to us. Previously they all came in through the serial interface."
fa0/0(10.1.1.1/24)------site1(10.1.1.2)----site3(10.1.1.3)etc...
With all the routers having routes to eachother, this is all that's needed! -- Burt "

Referencing the above thread from 2008, we are in almost the exact same scenario. We have old 1721 Cisco routers over T1 at each of 18 remote sites, connecting back to a Cisco 7204 VXR,all sites are aggregrated, logically separated at this HQ router. T3 / DS3 interface (does that sound correct?) - similar to this:
controller T3 1/0
clock source line
cablelength 10
t1 1 channel-group 0 timeslots 1-24
t1 2 channel-group 0 timeslots 1-24
t1 3 channel-group 0 timeslots 1-24
t1 4 channel-group 0 timeslots 1-24
t1 5 channel-group 0 timeslots 1-24
t1 6 channel-group 0 timeslots 1-24
t1 7 channel-group 0 timeslots 1-24
t1 8 channel-group 0 timeslots 1-24
t1 9 channel-group 0 timeslots 1-24
t1 10 channel-group 0 timeslots 1-24
t1 11 channel-group 0 timeslots 1-24
t1 12 channel-group 0 timeslots 1-24
t1 13 channel-group 0 timeslots 1-24
t1 14 channel-group 0 timeslots 1-24
t1 15 channel-group 0 timeslots 1-24
t1 16 channel-group 0 timeslots 1-24
t1 17 channel-group 0 timeslots 1-24
t1 18 channel-group 0 timeslots 1-24
#Current interfaces on main HQ look something like this:
interface FastEthernet0/0
description Ethernet connection subnet mask set to class b based on design
ip address 10.1.0.12 255.240.0.0
duplex full
!
interface Serial1/0/1:0
description T1 to Site 1 CKT ID #1
ip address 10.40.253.2 255.255.255.252
service-policy output VOIP_POLICY
!
interface Serial1/0/2:0
description T1 to Site #2 CKD ID #2
ip address 10.41.253.2 255.255.255.252
service-policy output VOIP_POLICY
!
1) We want our remote site IP structure to remain the same (voip phones, voice-mail switch at site, router at site).
2) Vendor is laying fiber to all sites, handing us RJ-45-type Ethernet at each site & at HQ.
3) HQ will have 200 Mb/sec link to aggregate all sites at the new 10 Mb/sec speed (per site).
4) This will be Metro-E.
5) We still want each site to keep the router and ip structure it has: i.e.,
Sample site-router config may look like this (10.40.20.1), (10.41.20.1), etc.:
no aaa new-model
memory-size iomem 25
ip cef
!
ip auth-proxy max-nodata-conns 3
ip admission max-nodata-conns 3
no ip dhcp use vrf connected
no ip dhcp conflict logging
ip dhcp excluded-address 10.40.20.1 10.40.20.149
ip dhcp excluded-address 10.40.20.251 10.40.20.254
!
ip dhcp pool 0
network 10.40.20.0 255.255.255.0
default-router 10.40.20.1
domain-name our-domain.com
dns-server 10.1.2.112 10.1.2.110
netbios-name-server 10.1.2.112 10.1.2.110
netbios-node-type h-node
option 4 ip 10.1.2.112
option 156 ascii "ftpservers=10.1.20.18,country=1,language=1"
lease 30
!
no ip domain lookup
!
class-map match-all SHORETEL_VOIP
match ip dscp ef
!
policy-map VOIP_POLICY
class SHORETEL_VOIP
priority percent 75
class class-default
fair-queue
random-detect
!
interface FastEthernet0
ip address 10.40.20.1 255.255.255.0
speed auto
!
interface Serial0
ip address 10.40.253.1 255.255.255.252
service-module t1 timeslots 1-24
service-module t1 remote-alarm-enable
!
router eigrp 1
network 10.0.0.0
no auto-summary
!
ip forward-protocol nd
!

[NOTE: The sites are /24 and HQ is /12]

6) All site routers give out DHCP to the VOIP phones, and we hope to keep it that way
7) All sites converge here at HQ to get out to the Internet over 50 Mb line to Internet.

[note - all IOS to be 12.4 or higher - have some that are 15.x]
8) New site routers will all be 1841's, with dual 100 Ethernet
9) New site HQ router will be 2800, ditto - dual GigE Ethernet

So, how do we keep our existing IP structure at the sites, with current 1721 routers replaced by 1841 routers, HQ router 7204 replaced by 2800 and logically separate the sites back at HQ? We do not want to have to change IP structure
on 18 sites, 400 workstations, 18 site servers, etc. with our newly-arriving MetroE connections. Is is possible to do
what we are trying? If so, how do we accomplish it? Any possibility to do "internal VLANs" on MetroE? If not, what is
your best suggestion? Previous-referenced post gave a vague idea of configuring the main router with "sites"
referenced on the HQ router "site-facing" interface. Would someone please provide more detail and an example (maybe from a live similar site setup?) REMOVE ANY PRIVATE INFO as I have done!
Thanks in advance for any help!

 

[findmefast]

WOW! 2 days and NO replies?
BUMP?

 

[findmefast]

Anybody have any thoughts on configuration for the above example?
Thanks in advance!

 

[imbadatthis]

yeah but im too lazy to explain it.....
basically you will have to do the following - and my apology for being brief on it.. just lazyness and im doing this all day ... so .. :

1 - setup Tunnel between Remote end and head end. Encrypted tunnels aren't too hard but basically you are creating tunnel interfaces.
2 - setup EIGRP (all cisco so nice and easy to setup EIGRP) and advertise the existing network routes to the main office.
3- go for lunch and call it a day .


for exmaple :

lets say your head end has the following after the Fiber is installed:

interface gi1/0/1 : 10.254.1.1

and your remote site has following interface : gig1/0/1 : 10.254.2.1



int tunnle11011 (go with a nice naming scheme so you know what site you are connected to...)
ip address 192.168.1.1 255.255.255.252
tunnel source gig 1/0/1
tunnel dest 10.254.2.1
!encryption stuff here..


at remote site:
tunnel1011:
ip address 192.168.1.2 255.255.252
tunnel source gig1/0/1
tunnel des 10.254.1.1
!encryption stuff


at both sites:
route eigrp 100
network 192.168.1.1 0.0.0.2
passive-int default
no passive-int tunn 1011


as long as your remote site's network is a connected interface it should start being advertised to your head end...



We must go always forward, not backward
always up, not down and always twirling twirling towards infinity.

 

[findmefast]

@imbadatthis (TechnicalUser) - Thank you!!!
I think that is WAY beyond what we want to do. Just want to get it working with 'basic routing' and simple sub-interfaces.

Here is sample:
[MADE up some numbers on a 'test' network, here they are]

[MAIN HQ ROUTER - 2800 series Cisco]

interface GigabitEthernet0/0
description Ethernet connection HQ Side
ip address 10.1.0.21 255.240.0.0
duplex full
speed auto
!
interface GigabitEthernet0/1
description Ethernet connection From the sites
no ip address
duplex full
speed auto
!
interface GigabitEthernet0/1.1
encapsulation dot1Q 1 native
ip address 10.16.253.6 255.255.255.252
!
interface GigabitEthernet0/1.2
encapsulation dot1Q 2
ip address 10.16.253.10 255.255.255.252
!
router eigrp 1
redistribute static
network 10.0.0.0
no auto-summary

[SITE ROUTER #1]

interface FastEthernet0/0
ip address 10.16.39.1 255.255.255.0
speed 100
full-duplex
no mop enabled
service-policy output VOIP_POLICY
!
interface FastEthernet0/1
ip address 10.16.253.5 255.255.255.252
speed 100
full-duplex
no mop enabled
service-policy output VOIP_POLICY
!
router eigrp 1
network 10.0.0.0
no auto-summary
no eigrp log-neighbor-change

[SITE ROUTER #2]
interface FastEthernet0/0
ip address 10.16.40.1 255.255.255.0
speed 100
full-duplex
no mop enabled
service-policy output VOIP_POLICY
!
interface FastEthernet0/1
ip address 10.16.253.9 255.255.255.252
speed 100
full-duplex
no mop enabled
service-policy output VOIP_POLICY
!
router eigrp 1
network 10.0.0.0
no auto-summary
no eigrp log-neighbor-change

Now, will the above work??? - seems to work with 1 router, in my test lab
OR, do I still need to do anything like a "dot1Q 1" on [SITE ROUTER #1] and "dot1Q 2" on [SITE ROUTER #2] ?
And WHEN is that "native" piece needed? Is it needed only 1 time, on the HQ router, or on each site router also?
Seems "native" only is needed on the very 1st sub-interface? Is that correct? Thanks!

 

[imbadatthis]

is your vendor honestly giving you point to point links on each site ?
i doubt it.. they are most likley going to give you MPLS connectivity through a VLAN .....

as for the dot1q ..


interface GigabitEthernet0/1.2 <--- sub interface means that you are connected to a TRUNK port on a switch on interface Gig 0/1
encapsulation dot1Q 2 <--- means that to peel off vlan 2 from the trunk and use the following interface configuration for it.


assuming MPLS network from vendor, they tell you what vlan to expect, or they just tell you that on their SED (carrier's edge device) you will be connected to port xx.. if they say VLAN then you have to use sub interfaces and dot1q tagging. if they say just use SED port xx then you dont need to use tagging or sub interfaces...

then yeah you can run eigrp between the various sites.. but you DO realize that all your traffic from all remote sites is traversing the ISP's network now and can be seen by anyone and anything ... NO SECURITY ...


We must go always forward, not backward
always up, not down and always twirling twirling towards infinity.

 

[findmefast]

@imbadatthis: Thank you again!!! Yes, I definitely realize the traffic potentially can be sniffed, without tunneling. Again, I thank you for the detail on the tunneling config, which is most likely what we will do but, in the mean time, we just wanted to see it work with an old 'test laptop' and then, once working, implement tunneling.

Also, the vendor apparently is doing something like QinQ - so that, in general, any VLANning on their part is transparent to us, and vice-versa. I said it is LIKE QinQ - the vendor does not provide details specifically, but that is the way they described it to us yesterday. It is very similar to a 'virtual point-to-point' but, again without vendor specifics, my guess to my boss yesterday was that it is some flavor of MPLS. Again, I only know enough to be dangerous, so thanks for any and all info you have given (or you may give) to help our 'small, limited-budget shop' get this going.

Tried links yesterday 'non-tunnelled' and, though they worked in my lab, i could get zero signal through the fiber - so, I will call vendor contact Monday and we will have them verify the line. Ours is a "mesh" between our 18 sites so, yes, they likely are using some sort of MPLS aggregation with VLANs inside their network. Ideally, and per the contract, even if the main HQ site goes down, due to the fact of it being a "mesh," all other sites still can talk to one another.

So, though I don't know their internal details, when I did setup the basic, non-tunneled link between 2 sites,
I was never able to get it to work. I even tried removing the router at the site and at HQ, and just making it a "plain old switched" connection - but no go.

They say the link/line/circuit is "up and hot" and, in general, the fiber transceiver shows that to be true but, since I don't know much more about testing routes over MetroE, does anyone know of any better ways I might be able to verify?
To me, if it worked in the lab, WITH routers, then it should work across the MetroE.

To clarify, we wanted to use the 'sub-interfaces' much like in the 'referenced previous thread' - i.e., keep all 18 sites separated (with or without VLANS - we don't care - just desire that logical separation). So, though we don't really want or care to use VLANS, we were (as stated in my orig post) wanting to keep the sites "as-is," with a 'site router' at each site, and no IP changes in the sites' router configs - i.e. 10.42.20.0 / 255.255.255.0 might be Site1 and 10.43.20.0 / 24 would be Site2 - again, I didn't know if that was possible, which is the crux of this whole exercise.

Then, of course, at HQ, something in the router would "magically" (network stuff is 'magic' to us 'systems' folks) look at the single "MetroE" ethernet drop that we have from the vendor at HQ and "magically separate" and/or pass the traffic that is coming from those 18 sites - and vice-versa - pass traffic destined properly back to each site. So, if I understand, I may be able to set up those sub-interfaces fine, even without any dot1Q/VLAN config, right? i.e., a plain old 'sub-interface' with an IP (just leave out the dot1Q stuff)?

So, if I understand correctly, I may or may not even have to do any VLAN tagging. As for 'trunking,' I'm still baffled as to what that means - network folks toss it around like it's a 'known' term, but it's not (not by system folks anyway).
i think it means bringing all traffic into an aggregated port, sort of - right? And then that port can be split up into VLANs as desired - that's at least the gist of my understanding of it.

To clarify a bit more:
HQ = 2800 cisco router with 2 Gig/e ports, one will be outbound to/from the sites (Gig0/1) the other is inside, onto our main switch backbone of HQ LAN (Gig0/0). At the site: similar - 1841 Cisco router, Fa0/1 outbound from the site back to HQ. Fa0/0 will be the internal Site LAN (i.e. 10.42.20.0 / 24).
Because at HQ they really only hand us fiber, we have to use fiber switch, just to take the port from Vendor to Ethernet.

[HQ - Backbone to router to switch, then onto Site-facing Fiber <--> to/from site]
[Site - Backbone to switch, to router, onto the Ethernet interface of vendor's HQ-facing fiber transceiver <--> to/from HQ]

I would be exceedingly grateful for any more ideas on getting basic connectivity from HQ to/from the site. FYI, the vendor did, indeed say that we don't have to (nor need to) configure any VLANning but that, if we do, it will be transparent to them (again, sounds much like this "QinQ" I've read about). Vendor also clarified that they have indeed configured the site links as 'tunnels,' so the security issue may not be as grand as thought, but definitely will be assessed.

Grateful beyond measure for all the assistance so far! (again, we are miniscule budget - cannot afford Cisco tech).

 

[imbadatthis]

maybe do a quick visio ?

you have alot of stuff in here.. and some of it is way off base and some of it is very confusing to me... of what you are trying to do ..

the 'magic' wont happen unless you configure it to happen..

if your vendor is suggesting that it will be a mesh then most likely you will not have to worry about trunks or vlans ..
however you still have to worry about routing which from what i saw on your org config it should be kosher..

again diagram.. doesn't have to be detailed.. but i would need the following to help you out :

HQ vendor facing address.
Site 1's vendor facing address.

on both devices:
sh ip route



We must go always forward, not backward
always up, not down and always twirling twirling towards infinity.

 

[insureme]

For what it's worth, my company runs on a MetroE provided by our Telco. We don't do any tunneling, although after reading what imbadatthis said, i might look into it. the routers at each location have a network local address on one side, and then all connect to the same Vlan on the other side as a transit network creating a star topology. for example all my routers use G0/0 as the Site2Site interface using a x.x.x.x/24 address. then for each site there is the G1/0 interface using y.y.y.y/16, and another site on z.z.z.z/16. The routing happens from the local networks to that central network provided via vlan from the Telco. EIGRP is an excellent idea as well, although all mine are static for the moment which give me more control over path determination.

 

[findmefast]

Ok, this is not Visio - just MS-Paint, believe it or not.
Diagram may make it more clear.

What would be helpful is someone with a "working config" to "sanitize" (take out your personal IP/data info - change it to fake data) and post the configs, so I can actually "see" them. That's what I did above - 'sanitized' configs, with personal info removed. It's great to say, "Here's the [theory] of how you do it." It's even more great to say, "Here's an example of our working setup (with personal info removed).

I don't have any "vendor-facing" addresses "from" the vendor - we are told we don't need any of that - it's supposed to be moot, since it's much like a "wire between sites." But, if you mean my "site-facing" address; i.e., the ip address of the interface that is outbound to the site(s), I've included that. Link below takes you to the image of an example network that is similar to what we want to do:

Link

 

[insureme]

Assuming your demarcation is at your site routers, your ip configuration already makes sense for the most part. As i said in my example. the network you are using for transit is probably just a vlan from your provider. so take your addresses 10.16.253.x and widen the mask from 252 to 248, or more if needed. drop the sub-interfaces on the HQ router and replace with just a single IP address (10.16.253.6, and on each device configure static routes, i.e.
Site 1
ip route 10.1.0.21/12 10.16.253.6
ip route 10.43.20.0/24 10.16.253.9

Site 2
ip route 10.1.0.21/12 10.16.253.6
ip route 10.42.20.0/24 10.16.253.5

HQ
ip route 10.42.20.0/24 10.16.253.5
ip route 10.43.20.0/24 10.16.253.9

Sorry i don't have time to give you a sanitized configuration as you asking for. If you don't mind my saying this is a somewhat complicated setup, and if you are not real familiar with networking, routing, and switching you may want to employ professional assistance.

 

[imbadatthis]

I don't have any "vendor-facing" addresses "from" the vendor - we are told we don't need any of that - it's supposed to be moot, since it's much like a "wire between sites." But, if you mean my "site-facing" address; i.e., the ip address of the interface that is outbound to the site(s), I've included that. Link below takes you to the image of an example network that is similar to what we

if that is truly the case, then all you'll have to do is get rid of all sub interfaces, change the mask at all sites to a /24 and theoretically you should be able to ping from one site to another site with out issues..


i would really at least consult a person in your area that does networking..
if you are in vancouver, or edmonton i know some folks that dont have high rates but do good work for what you need,otherwise, I would call around and see...

good luck.


We must go always forward, not backward
always up, not down and always twirling twirling towards infinity.

 

[findmefast]

Many thanks to you guys! It works - both with, and without, the sub-interfaces.

No VLANs needed but, I suppose we can set those up as needed, since it's all transparent to the vendor.

Only 2 issues remain:
1) We must have sub-interfaces or equivalent, as stated originally - so, if we can't have sub-interfaces, can we get a "plug-in" physical module with 18+ ports in it and then set needed QoS settings on those physical interfaces, like we did in ye olden days on the Serial sub-interfaces? What are our options? I don't mind having a physical card with multiple ports in it, if that's what's needed to configure it.

I hope everyone can understand that I have the same situation as the OP in the 'reference link' I gave at the top of this thread: We need to have separate interfaces (physical or logical [i.e., sub-interfaces] - that part doesn't matter), so that we can have proper tracking and separation of stats (i.e., 'probes' for network monitoring type stuff) for each site. But, if we're "over-thinking" it, and if static routes will still allow network monitoring and separation of the individual sites, that might be possible - I just don't really like the idea of static routes.

2) We need to be able to get VOIP QoS working on all those 18 sites - and, though the above scenarios work for normal network traffic, the Cisco 2800 router at HQ does not allow setting QoS on Ethernet sub-interfaces. Some sort of CBWFQ error. Here's the version of the HQ router - I know that 15.x and higher is out now, but we have no support, and I don't think it has any changes in allowing sub-interfaces QoS settings on Ethernet ports.
Cisco IOS Software, 2800 Software (C2800NM-SPSERVICESK9-M), Version 12.4(24)T3,
RELEASE SOFTWARE (fc2)

When I set this up yesterday, data is working great - about 80Mb/sec compared to the old T1 1.544Mb/sec - we're now going over Fiber and have a CIR (Committed Info Rate?) of 10Mb/sec, so seeing 70 to 80 Mb/sec was mind-blowing!
But the phones did not work - they were sporadic - one side could hear, but other side broke up terribly.

Policy on HQ router was like this on the old 7204VXR router:

class-map match-any SHORETEL_CONTROL
match access-group 101
match ip dscp af31
class-map match-all SHORETEL_VOIP
match ip dscp ef
match protocol rtp audio
!
!
policy-map SHORETEL_VOIP
class class-default
set dscp default
fair-queue
policy-map VOIP_POLICY
class SHORETEL_VOIP
priority percent 25
set dscp ef
class SHORETEL_CONTROL
bandwidth percent 10
set dscp af31
class class-default
set dscp default
fair-queue

interface FastEthernet0/0
description Ethernet connection subnet mask set to class b based on design
ip address 10.1.0.21 255.240.0.0
duplex full
!
interface Serial1/0/1:0
description Connection to Site 1
ip address 10.27.253.2 255.255.255.252
service-policy output VOIP_POLICY
!
interface Serial1/0/2:0
description Connection to Site 2
ip address 10.36.253.2 255.255.255.252
service-policy output VOIP_POLICY

But, on the new 2800 router ("new" to us), when applying service-policy to sub-interfaces, I get this error:
CBWFQ : Not supported on sub-interfaces

And, there are tons of articles on something called "nesting" for applying QoS to Ethernet sub-interfaces, that is a work-around - does anyone have a LIVE, "cleansed" example of how to do that (with something similar to our above QoS [VOIP_POLICY]) settings)? I saw examples, some mentioning "parent - child policies," but it's unclear how to convert the above older 7204 working settings to the newer "nested policy" settings that appear to be needed on Gig Ethernet sub-interfaces on a 2800 router.

BUT, if getting 18 'physical interfaces' in a plug-in card will work, then we will do that, and be done with it.

Any tips & direction would be greatly appreciated.

Thanks!

 

[imbadatthis]

doubting that your vendor will provide you 18 separate Fiber cuts (or vlans) .. you are left with what i said on top .. you have to build tunnels ...


We must go always forward, not backward
always up, not down and always twirling twirling towards infinity.

 

[findmefast]

Thanks. I guess tunnels or "nested QoS" ?
No, I didn't expect 18 fibers, but if the one fiber can be directed to the 18 switch ports that will be built into a module?
Is that even possible? Each physical interface would be like a sub?

Heirarchical queuing and/or nested:
Link

I meant to say whatever "router port" (i.e. 23/24 port router module) that we are putting into the 2800 router, that should allow us to set individual QoS on each port, run each port up to a port on the switch that leads to the outbound "site-facing" fiber, and shouldn't that do the trick? In other words, it's like having 24 routers in one, as long as we feed to the switch that feeds to the "site-facing fiber" - right?

 

[findmefast]

Scratch that! Never mind - the module someone mentioned was indeed a "switch" module.
Go figure - so it would not work, as "imbadatthis" said, unless for sure there were such a thing as an 18+ port actual "router" module.

So, yes, we are down to "tunneling" and/or "nested QoS". Argh! Or, just don't use routers at our HQ site, and be happy with a "switched" configuration. ?