See Bandwidth License Useage in Juniper vMX


Juniper vMX features are managed via a software license. There are three levels of licensing along with the overall bandwidth license. Also, licenses are additive; if you add more than one license to a vMX router then it will add all of the licenses together. This is a great way to manage to growth as you continue to expand the usage of your vMX router.

The following shows you how to verify the software licenses installed and the total bandwidth of the vMX router.


Determine the bandwidth license usage in Juniper vMX and configured bandwidth in the packet forwarding engine (PFE).

Components used

  1. VMWare ESXi 6.0 Update 2
  2. Juniper vMX 16.1R7

These instructions also work with vMX deployed on KVM or other virtualization platforms. The commands are only dependant on junos.

Verify Commands

The “show system license” command displays the licenses installed and their current usage. In the example below the system has two license for a total of 100 mb of bandwidth.

 show system license   
show system license example output:

Use the “show pfe statistics traffic bandwidth” to show the actual bandwidth being used. This shows the configured bandwidth based off of the licenses along with the current and average bandwidth being used.

show pfe statistics traffic bandwidth
show pfe statistics traffic bandwidth output:


If you combine the same command with the pipe command , the “refresh” command and a value in seconds,  then you can set up a nice little bandwidth monitor. This is very handy if you are troubleshooting and you think you’re out of bandwidth, licensing has kicked in, and you’re seeing some impact to traffic outside normal troubleshooting.

show pfe statistics traffic bandwidth | refresh 1
show pfe statistics traffic bandwidth | refresh 1 output:


I hope these commands help you out!

Cisco ASA to Juniper SRX Migration

Below is an excellent resource for engineers who are already familiar with Cisco ASA firewalls and want a no nonsense guide to covert over to Juniper’s SRX next generation firewall. Rather than go over security concepts you’re already familiar with it mainly goes over the common things that people need to migrate. It also does an excellent job of highlighting some of the architectural differences between the two platforms.

Cisco ASA to Juniper SRX Migration Book

VMware vSphere 6, Update 2 Home Lab

I’ve been running various VMware ESXi home configurations for many years. Until recently, I’ve been using Apple Mac Minis and kept hoping Apple would get around to updating them to run additional memory in their next release. Alas, no such luck. I used VirtuallyGhetto’s guide back in the day to set them up. This is a great setup. It’s painless to configure, quiet, and has low power requirements. Here’s that guide linked below.

VirtuallyGhetto’s Mac Mini ESXi guide

Unfortunately, these days, I need something with a lot more memory and processing power. But I still want something small, easy to setup, quiet, and easy on the power bill.

After some research, I decided to use Shuttle PC’s SH170R6. It supports 64 GB of RAM, the latest Intel Skylake i7 four core 6700k processor,  a built in Intel i219LM network adapter, and an integrated video port. The power supply is only 300 watts and the CPU is a 95 watt processor. The old mac minis use 85 watts each. So my power will be more than my old mac minis but the processing power is much more and has four times as much RAM so I think it’s an excellent tradeoff. The setup is fully recognized without any additional software drivers with VMWare ESXi 6, update 2. Unfortunately, because of the newer Skylake chipset, you can’t run older versions of VMWare on this platform.

Shuttle SH170R6 on Newegg

Also, after doing some additional research, I discovered it’s got an M.2 port so I decided to use it to load ESXi onto a small M.2 drive it as opposed to booting ESXi off of an external usb drive.

The labs I’m doing  these days are focused on networking so I added an additional four port Intel I350-T4 PCI-E network adapter. This isn’t the cheapest card available but it’s recognized natively without drivers and supports IO-SRV. I’m building a lot network function virtualization labs and want to experiment with the support of IO-SRV in VMware vSphere. These ports will be trunked to my three Juniper EX2200-Cs that are configured into a single virtual chassis. I’m using the built in Intel adapter as the VMware management interface.

Intel I350-4T on Newegg

Here’s some photos of the external layout and form factor of the Shuttle SH170R6.




Here’s what it looks like inside without the drive cage installed. It’s got plenty of room to install the components. A nice touch is most of the cables are already plugged in from the factory. It makes for a very clean setup.


Another cool feature is the Integrated Cooling Engine 2 (ICE 2) Heat pipe technology by Shuttle. It comes with the case and does an excellent job of keeping the CPU cool.


Here’s what it looks like outside of the case. It’s a very simple design.


So for storage, I’m using an ADATA M.2 128 GB card and 960 GB SSD. The M.2 card is for ESXi and the SSD is for VM storage. The ADATA is much bigger than what’s needed for the ESXi OS but it was only 50 bucks on newegg.

Also, to mount the SSD in the case I picked up this Sabrent adapter. It’s only 1o bucks and it allows you mount two 2.5 inch SSDs in an 3.5 drive bay. I might expand the internal storage in the future. It’s nice to have options.

Sabrent 3.5-Inch to x2 SSD / 2.5-Inch Internal Hard Drive Mounting Kit


Here’s the SSD drive mounted in the Sabrent adapter.


Here’s the M.2 drive mounted in the case. It sits flat in the case so if you have a longer PCI-E card it’ll have good clearance and not interfere.


I got G.Skill Ripjaws RAM for this system.

G.SKILL Ripjaws V Series 64GB (4 x 16GB) 288-Pin DDR4 SDRAM on Newegg


The Shuttle comes with thermal paste so it makes it a snap to get the cpu mounted with the heat pipe heat sink.


Here’s what it looks like with the CPU and heat pipe heat sink mounted. Nothing to see here!


Here’s all of the components installed except drive cage.


Here’s the layout once the drive cage is installed.


Here’s what two Shuttles look like installed in my lab. In this setup, I’ve got 128 GB of RAM and 8 i7 cores to play with. You can also see my old mac minis in the center.


I’m really happy with this setup. It’s dead quiet, even under full CPU load. These days, my two QNAP network storage arrays are the loudest things in my rack setup. The three Juniper EX2200-C POE switches are fanless so they are silent as well.

The full lab along with the Juniper switches allow for excellent, close to real world, virtualization and networking scenarios. The Juniper EX2200s support full Junos OS features such as VLAN trunking, virtual chassis, LAG, VLANs, private VLANs, PoE, QoS, and layer 3 routing protocols.

Setting up Juniper vMX on vSphere vDS

Juniper has a great “Getting Started” guide on setting up vMX on VMware vSphere. In the “Installing vMX with OVA files” section of the guide, it only uses the standard vswitch in the setup.

Installing vMX on VMWare Guide

So how do we setup vMX on a Virtual Distributed switch? Unfortunately, the “Getting Started” guide won’t help you. So I’ve created this guide to help you out. This guide assumes you’ve already setup a VDS already in your VMware environment. Below are two links that go over the basics of setting up the VMware vSphere Distributed Switch if you haven’t done that yet.

Part 1: Getting Starting with the VMware vSphere Distributed Switch

Part 2: Getting Starting with the VMware vSphere Distributed Switch

This guide was created using VMware vSphere 6.0 Update 2 via the web console. While the official documentation for vMX supports VMware 5.5, 6.0 works fine for lab environments.

So rather than repeat the entire “Getting Started” guide, I’m just going to go over the differences starting from the “Setting Up the Network” section of the guide.

Once you complete this section you can continue on the “Getting Starting” guide at the “Deploying the VCP VM” section.

  1. After logging into the web interface, click on the Networking section.


2. Click on the root of the virtual distributed switch, then the “Getting Started” tab, and then the “Create a new port group” section to start creating a port group.

VM port groups are a way that we can create logical rules around the virtual ports that are made available to VMs. It’s common to create a port group for each VLAN and network subnet that you want to present to your VMs. Port groups also can be used to create 802.1Q trunks.


As shown in the drawing below we are first creating the management port group. This is just a personal preference, but I normally name the port groups after the VLANS that they are part of. In the case of the management port group, you can just make one and associate all of your VMX’s first interface to the same VLAN. You don’t need to make a separate management port group for each router.


3. As stated earlier, give the management port group a descriptive name. In this example shown below, I’m using “VLAN350-Juniper-Mgmt”. Click next.


4. Change the VLAN type to “VLAN”, give it a VLAN ID, and check the “Customize default policies configuration”. In this example, I gave it the VLAN ID of 350. Click next.


5. This step is critical.  In the security settings, “Promiscuous Mode”, “MAC Address Changes”, and “Forged transmits” need to be set to “Accept”. If you don’t do this you won’t be able to get into the management interface of the vMXs that you create later.


Next step we need to create the internal bridge between the VFP virtual machine and the VCP virtual machine, as shown in the illustration below.

This is just like the internal connection of a real hardware MX router that separates the control plane from the data forwarding plane. In this case, the Control Plane is the VCP virtual machine and the VFP virtual machine is the data forwarding plane. The VCP only has two connections. One is the management interface we created earlier and the other is the next interface we are going to create. This  interface has to be unique for each pair of VCP/VFP virtual machines.

The VFP virtual machine can have up to ten interfaces. The last eight of them will be revenue ports for routing and bridging traffic but the first two interfaces are the management port and the internal port to connect to the VCP virtual machine.

FYI, the ten interface limit is a limitation of VMWare. Linux KVM can scale to 23 port per vMX.


6. We start the process the same way we created the other port group. In this case, I’ve given it the name of “VLAN351-R1-br-int”.


7. Change the VLAN type to “VLAN”, give it a VLAN ID, and check the “Customize default policies configuration”. In this example, I gave it the VLAN ID of 351. Click next.


8. Just as in the previous example, this step is critical.  In the security settings, “Promiscuous Mode”, “MAC Address Changes”, and “Forged transmits” need to be set to “Accept”. If you don’t do this VCP and the VFP virtual machines won’t be able to communicate with each other and you’ll have no interfaces because the VFP won’t register to the VCP via the internal bridge we are building with this port group.


Final notes, before you continue on to the “Getting Starting” guide at the “Deploying the VCP VM” section.

  • The Management port group can be the same for all of the vMXs. But the internal port group MUST be unique for each pair of VCP/VFP virtual machines. If two vMXs have the same port group as their internal bridge, neither will come up properly.
  • When you create the other eight ports on the VFP virtual machine, you can use the same process as described above. “Promiscuous Mode”, “MAC Address Changes”, and “Forged transmits” need to be set to “Accept”.
  • Also, if you want to play with 802.1Q trunking you can setup a port group as  a trunk instead of a VLAN. Then you can play around with trunking vMXs together like you would traditional switches. There’s a lot of flexibility.
  • This is just a personal preference, but I also create a port group and name it “disabled” and go into the “Miscellaneous” section of the port group and set “Block all ports” to “Yes” as shown below. Then any port that I’m not actively using on the VFP virtual machine I normally put in this port group.


I hope this helps you. Please feel free to leave me comments, questions, or corrections.

Tim Gregory’s EVPN Examples

Tim Gregory has some great, simple examples of EVPN on Juniper routers. All of his examples use MX routers and EX4200 switches.

The four examples below highlight the simplicity of EVPN and how it addresses the shortcomings of VPLS in the Data Center Interconnect (DCI) role with an open standard.

Here’s some of the benefits of EVPN over other legacy and/or proprietary DCI technologies:

  1. Flexible Data Plane options
    • VxLAN or MPLS
  2. All-Active Forwarding
    • Scale up and out your connections between data centers with additional routers and connections; No wasted connections
  3. Control Plane Learning for Mac Addresses
    • Eliminates flooding of BUM traffic / improve Learning / eliminates loops
    • Fast convergence; usually sub-second mac address adds, withdrawals, and moves
    • Faster reconvergence when a Virtual Machine moves
  4. Combined L2 and L3 Services
    • Eliminate the trombone effect for layer 3
      • Distributed L3 gateway for Virtual Machine Traffic Optimization (VMTO)
    • Supports Integrated Routing & Bridging natively
  5. Open standards interoperability;
    • Juniper, Cisco, and others support it

Here’s the links for Tim Gregory’s articles:

EVPN – The Basics

EVPN Inter-VLAN routing and Mobility

EVPN All Active Multihoming

EVPN Single Active Redundancy

Later, when I get a chance, I’m going to post some modified examples of these configurations running running fully virtualized on VMWare vSphere just using Juniper vMXs.

Using vMX free trial licenses allow you to build a lab around EVPN for free and test it out.