Create great junos labs with Logical System and Tunnels. Works in vMX too!

Here’s a great article from 2012 on Junos logical systems and logical tunnel interfaces. MX routers support 15 logical systems per router and when you combine this with logical tunnel interfaces, you can make extremely complex lab networks with one or more MX routers. Here’s the article below along with some additional information about logical tunnel interfaces.

JUNOS Logical-System: A Great Way To Learn Junos & Prepare For Certification

Configuring Logical Tunnel Interfaces

Also, you don’t have an MX router laying around? No problem! This is fully supported on Juniper’s vMX.

Below is an example to enable tunnel services within a vMX router.

chassis {

    fpc 0 {

        pic 0 {

            tunnel-services {

                bandwidth 10g;


            interface-type ge;          

            number-of-ports 8;




    network-services enhanced-ip;


Here’s an example topology I created with logical systems and logical tunnel interfaces between the logical systems. In this example, there are two vMX routers with five logical systems each. This is like having ten routers total. Within the MXs, the logical systems are interconnected via logical tunnel interfaces which saves the ge interfaces for just interconnecting the vMXs together. That’s important due to the current limitation of 8 useable revenue interfaces on vMX routers.


Have fun making great labs with this!

EVPN – Basic Layer Two Extension Lab, Virtual Lab Example


This is a basic introduction to EVPN and why it’s a better alternative to other Data Center Interconnect (DCI) technologies such as VPLS/MPLS. Here I’m going to repeat some information from Tim Gregory’s posting.

####insert EVPN stuff here, additional detailed info added later ####

Brief description of EVPN and it’s benefits

####insert EVPN stuff here, additional detailed info added later ####

VMware/vMX Lab Example

This lab example is based off of Tim Gregory’s lab example linked below.

Tim Gregory’s EVPN Basic Example

Now Tim’s article is great, but most people don’t have a bunch of Juniper MX routers laying around, so I’ve created the same configuration virtually using Juniper vMXs running on VMware ESXi. I also added some client linux VMs to log into to and test to connectivity between sites. In addtion, having the linux VMs allow for a excellent way to demonstrate data center technologies such as VMware vMotion between data centers combined with EVPN.

Historically speaking, you normally can’t stretch a VMware vSphere between two or more WAN separated data centers. With the advent of VMware vSphere 6.0 you can vMotion between layer 3 separated data centers but it requires changes to your VMware environment and some technical limitations still exist. Just using EVPN doesn’t require any changes a production VMware vSphere environment. It works for all versions of VMware vSphere since it’s hides the networking complexity from VMware.

Lab Diagram

Lab Components used

  1. VMWare ESXi 6.0 Update 2
  2. VMware vCenter Server Appliance 6.0.0
  3. Virtual Distributed Switches
  4. Six Juniper vMX routers running 16.1R1.7 code
  5. vMX Demo license
  6. Three Linux Desktop VMs

vMX router configurations

The link below is a zip file containing the example configurations for the vMX routers used. Also, the account information is below.

root password: lab123

admin account name: admin

admin account password: lab123


General Lab Configuration Notes

Looking at the lab drawing above, notice the connections between the routers. Since we are using VMware, we need to make connections between the routers using VMware port groups. In VMware we give each port group a unique VLAN number and we create one port group between each pair of devices. This guarantees each network segment between the devices are isolated and no other devices are seen.

Also, we need to remember to make sure in the “Polices” section of the port group that the “Promiscuous Mode” option is set to “Accept”. If this isn’t set, then you won’t see any traffic between the vMX routers.


Security Setting Example for Port Groups

Also, another note you need to know, is the vNICs associated between the FPC VM and the actual interfaces inside the vMX router are randomly assigned after the first three vNICs. What does this mean? Remember the first two vNICs are for management and the internal connection to VCP VM. The other possible eight vNICs can be used for connections to other devices. But you need to  match the MAC address of the vNIC to the MAC address of the vMX interface.

Here’s an example of the R3-P1 FPC VM in the two pictures below. Notice the vNICs aren’t listed in same order that they are on the vMX router. In this example, vNIC 4 is associated with ge-0/0/2 inside the vMX router. ge-0/0/2 is the third ge interface in the vMX but it’s the fourth vNIC inside the VM.


VM vNIC Example

Luckily we can track down the correct interface with a quick “show interface” command on the vMX and compare the hardware address inside the vMX to the MAC address inside VMware.


Show Interface Example

Also, as a personal preference, I normally name my port groups with the VLAN number first and then what segment it’s supposed to represent. For example, VLAN444-R1-PE1-to-R3-P1 port group uses VLAN 444 and is used between router R1-PE1 and router R3-P1.

Technical limitations with VMware vSwitch and Work Arounds

Now Tim Gregory’s lab example uses physical EX4200 switches to connect VLANs to the MX routers. Let’s keep it virtual. I’m using VMware’s virtual distributed switch as my switch. This switch is distributed between all of my ESXi hosts. It simplifies my virtualization environment and keeps the vdswitch consistent between all of my hosts. But the example lab environment needs to have the same VLAN on all three EX4200 switches. In VMware, if I create three different port groups on the same vdswitch and give them the same VLAN ID, the VMs will see each other at layer two because they on the same vdswitch. That’s a big problem in the lab example when we convert it to VMware. That’s because we want the linux VMs to only connect to their vMX PE router ge interfaces so we can truly show EVPN 2 layer connectivity and mac  address mobility.

One option I have is to create three different vdswitches but it complicates my VMware environment because I have to associate different ESXi host NICs to each vdswitch I create. Another simpler option, is to create port groups as I normally do, each with a unique VLAN ID, them use the vMX PE routers to untag the VLAN ID when the traffic comes in from the virtual LAN segment, and put the traffic in the correct VLAN inside the PE routers. This is what i’m doing in this lab example.

For example, for the port group connecting the linux hosts to the R1-PE router has a VLAN ID of 461 but within the vMX I don’t care about the VLAN tag and I drop the traffic into VLAN 458 via the bridging-domain subsection of the routing-instance.

Here’s the configuration of the port group in VMware.


Example Port Group in VMware

Here’s an excerpt of the R1-PE1 router’s interfaces configuration. In this example, ge-0/0/0 and ge-0/0/2 are setup as switching ports. These are the ports I connect the linux desktop VMs to the vMX.


R1-PE1 ge Interface Example

Here’s an excerpt of the R1-PE1 router’s routing instances configuration. In this example, ge-0/0/0 and ge-0/0/2 are in the bridge domain and are associated to VLAN 458. It’s almost the same setup on R2-PE2 and R5-PE3 PE routers.

evpn-r1-pe1-routing-instance-configR1-PE1 routing-instance Example

Important Note: Routing traffic between more than one ESXi host.

Another thing to note is for port groups to pass VLAN traffic between two or more ESXi hosts you need to make sure the VLANS also exist on the physical switch connected to the ESXi hosts. Also, the switch ports that the ESXi NICs are connected to are configured as Trunk ports and that the required VLANs are allowed on the ports.

Additional Troubleshooting and Verification Commands

Tim Gregory’s article has lots of good commands to verify that your configuration is working. Beyond what he’s providing here’s some additional things I usually use.

show bridge mac-table

I use the “show bridge mac-table” to verify all of the layer 2 mac addresses coming from both the local linux VMs and the remote  linux VMs connected to the other PE vMX routers.

In the example below, you can see one linux VM attached to ge-0/0/0.0 and another one connected to ge-0/0/2.0. You’ll also notice two other mac addresses with no logical interface listed. These are from the EVPN layer two LAN segments and are from the other two PE routers. These MAC addresses are from the other two linux desktops connected to those PE routers.


“show bridge mac-table” Example

On the the terminal of the linux desktop attached to R1-PE1, if I run the “ifconfig | grep HW” command, I can see the local MAC address of the linux machine and I can verify that it’s the same MAC address that is shown on the vMX router on port ge-0/0/0.0.


“ifconfig | grep HW” Example

I can use a combination of these two commands on all of the PE routers and the linux machines I can track, at layer 2 on the PEs, which linux machines are connected and where.


I this helps you out in your own labs. 🙂

See Bandwidth License Useage in Juniper vMX


Juniper vMX features are managed via a software license. There are three levels of licensing along with the overall bandwidth license. Also, licenses are additive; if you add more than one license to a vMX router then it will add all of the licenses together. This is a great way to manage to growth as you continue to expand the usage of your vMX router.

The following shows you how to verify the software licenses installed and the total bandwidth of the vMX router.


Determine the bandwidth license usage in Juniper vMX and configured bandwidth in the packet forwarding engine (PFE).

Components used

  1. VMWare ESXi 6.0 Update 2
  2. Juniper vMX 16.1R7

These instructions also work with vMX deployed on KVM or other virtualization platforms. The commands are only dependant on junos.

Verify Commands

The “show system license” command displays the licenses installed and their current usage. In the example below the system has two license for a total of 100 mb of bandwidth.

 show system license   
show system license example output:

Use the “show pfe statistics traffic bandwidth” to show the actual bandwidth being used. This shows the configured bandwidth based off of the licenses along with the current and average bandwidth being used.

show pfe statistics traffic bandwidth
show pfe statistics traffic bandwidth output:


If you combine the same command with the pipe command , the “refresh” command and a value in seconds,  then you can set up a nice little bandwidth monitor. This is very handy if you are troubleshooting and you think you’re out of bandwidth, licensing has kicked in, and you’re seeing some impact to traffic outside normal troubleshooting.

show pfe statistics traffic bandwidth | refresh 1
show pfe statistics traffic bandwidth | refresh 1 output:


I hope these commands help you out!