APIC-EM PNP deployment on Cisco 892FSP with 200+ MPLS sites

• Curently trying to deploy 200+ remote MPLS L3 VPN sites for a customer with APIC-EM PNP based on Cisco 892FSP. APIC-EM is running 1.4 and is configured with templates and everything looks fine. Each 200+ sites will have the following topology: 
•  VLAN 3001 is used for management and the ISP CPE has an helper address on this VLAN pointing to the customer DHCP configured with option 43 in order for the customer CPE to be able to communicate with the APIC-EM controller.
•  On the customer CPE, the trunk is configured with L3 subinterfaces. All Cisco 892FSP are running on 15.5(3)M4a software version which should support PnP . Checking the APIC-EM PNP compatibility matrix. The router has been reset to factory default following APIC-EM PNP configuration guide.
•  The issue already starts when you boot a 892FSP which stops on the initial configuration dialog, where user has to choose yes or no:
• It looks like you are hitting the following bug: https://bst.cloudapps.cisco.com/bugsearch/bug/CSCuu93989/?referring_site=bugquickviewredir

A couple of things:

1) The message you see is normal, it does not mean PnP is not working.  The process happens under the covers

2) In order to do what you need, you will need dynamic trunking on the uplink port.... but routers do not support DTP.

3) You will also need CDP to negotiate the startup vlan on the 892.  but it would need to create the sub interface, not just a vlan.

 Able to get a connection to come up using DTP on the switch and native vlan == management vlan on the switch.  The challenge is that if you push a config that contains sub interfaces, there is no way to advertise trunking support from the router.  This is important as you need to use DTP to signal to the switch to move to trunking mode.

  the best solution is to use a USB key with a small bootstrap config (i.e. just the dot1.q of the management interface).   You can even leave on DHCP and the PnP process would do the rest.

Or try the following without using USB.

 Force the switch to trunk and make the management vlan the native vlan, then the router will be able to communicate to the PnP server (and use DHCP).

 You can then push a config down to the router to configure the management interface as management.  you can also move to a static IP at the same time.  you will need to do a "no ip address on the router WAN interface".

 On switch: (NOTE vlan 14 is my management vlan)

3850-core#show run int g1/0/7

Building configuration...

 Current configuration : 126 bytes

!

interface GigabitEthernet1/0/7

description link to ZTD router

switchport trunk native vlan 14

switchport mode trunk

end

 Then push the following config to the router via PnP

interface GigabitEthernet0/0

no ip address

interface GigabitEthernet0/0.14

encapsulation dot1Q 14 native

ip address 10.10.14.100 255.255.255.0

end

 The IP address could be DHCP (it would get another IP address as different MAC), or statically defined.

 The only challenge is you need to have the management vlan as the native vlan.

You are right, the PNP process is now happening. The issue was that the 892 wasn´t getting an IP address from the DHCP.

 The ISP CPE router is configured as follows ( I use another 800 for test purpose).Not configured the pnp startup-vlan command on the ISP CPE.

!

interface FastEthernet5

description #PNP-AGENT#

switchport trunk native vlan 3000

switchport mode trunk

!

interface Vlan3000

description # MGT #

ip address 10.250.148.1 255.255.255.252

ip helper-address 10.9.100.70

end

!

The Customer CE is connected towards the ISP CE on a routed port (G8). The PNP configuration we want to push from APIC-EM is the following on this port:

!

interface GigabitEthernet8

description # WAN #

media-type rj45

no shut

!

ip route 0.0.0.0 0.0.0.0 10.250.${WAN-LOKATION_ID}.1 name APIC-EM-PNP

!

interface g8.3000

description # MGT #

encapsulation dot1Q 3000 native

ip address 10.250.${WAN-LOKATION_ID}.2 255.255.255.252

no shut

!

interface g8.3001

description # ADM #

encapsulation dot1Q 3001

vrf forwarding ADM

ip address 10.250.${WAN-LOKATION_ID}.6 255.255.255.252

no shut

!

etc... Until 3008

!

When booting, the Customer CE router gets the APIC-EM info from DHCP and also contact the APIC-EM controller.

It looks like everything goes well on the APIC-EM controller as it goes from "pending" to "deploying config" but then it goes stuck in this state and after a couple of minutes and then it changes to "error" state.

From the logs of  customer CE router, it looks like all the configuration has been applied by APIC-EM. The only thing which is missing is the IP address on the g8.3000 subinterface . Otherwise all the config has been applied successfully.

!

ip route 0.0.0.0 0.0.0.0 10.250.148.1 name APIC-EM-PNP

!

interface GigabitEthernet8

description # WAN #

ip address dhcp

duplex auto

speed auto

media-type rj45

!

interface GigabitEthernet8.3000

description # MGT #

encapsulation dot1Q 3000 native

!

interface GigabitEthernet8.3001

description # ADM #

encapsulation dot1Q 3001

vrf forwarding ADM

ip address 10.250.148.6 255.255.255.252

!

etc...3008

!

Is that because PNP IOS agent is not supported on routed subinterface.

The  issue is that when you boot router, it gets an IP address via DHCP on int Gig8.

 When you push the profile, it also tries to assign an IP address out of the same range in as the DHCP address.  As you know you cannot have the same network assigned on two different router interfaces.

Try to include the following

int g8

no ip address

 You would be successful.  That would remove the DHCP associated address, and allow you to use the manually assigned IP address in the template.

 After having configured the following in the template the router gets provisioned successfully:

int g8

no ip address

• Another question regarding the provisioning. Using a project and then a template to provision the router. So the customer as to add the 200 devices under the project and fill in the different parameters in the configuration template. The template as 7 variables as you see here:

•  would like to know if there is a more efficient way to do this process (scripting or bulk import).
• will it be possible to use an excel sheet as bulk import including the above variables, the project name, the configuration for each location and so everything can get created from the excel sheet automatically.

For the 920, try the "I wish this page would" on the bottom left hand corner.  That will send an email to the product owners.

 The bulk import of template variables is in the next release i think. 

 It is possible to script this as well via the REST API.  Check the blog post that contains the API calls to do this.  APIC-EM 1.3 Update – Part 1 - PnP Templates