Forum Discussion

Anonymous's avatar
Anonymous
9 years ago

Juniper Netflow configuration examples

Like most of you, I have a long list of "To-do's" in our LogicMonitor deployment.  One that I just recently crossed off is capturing Netflow.  More accurately, capturing sFlow from Juniper EX/QFX switches.  It's worth noting that the actual implementation was surprisingly easy, aided by LM's netflow doc page and Juniper's references (see links below).  I've included the needed Juniper commands for those of you who are also in non-Cisco environments.  I can't stress enough though that you pay close attention to LM's suggested best-practices, carefully consider Juniper's caveats, and put a lot of effort into planning the details of your deployment (which switches, which physical interfaces, etc) so that you understand what data you are actually getting from sFlow and how it is being delivered.  Remember: more data at your disposal does you no good if you can't place it in proper context.

###Global sflow enable, globally define polling interval and sampling rate,
      define sflow source address and agent id; define the sflow collector and export port
set protocols sflow agent-id 192.168.254.2
set protocols sflow polling-interval 1
set protocols sflow sample-rate ingress 100
set protocols sflow sample-rate egress 100
set protocols sflow source-ip 192.168.254.2
set protocols sflow collector 192.168.1.2 udp-port 6343

###enable sflow sampling on individual switch ports (note: polling and sampling values set here are not required and will override global values)
set protocols sflow interfaces ge-0/0/9.0 polling-interval 1
set protocols sflow interfaces ge-0/0/9.0 sample-rate ingress 100
set protocols sflow interfaces ge-0/0/9.0 sample-rate egress 100
set protocols sflow interfaces ge-0/0/11.0 polling-interval 1
set protocols sflow interfaces ge-0/0/11.0 sample-rate ingress 100
set protocols sflow interfaces ge-0/0/11.0 sample-rate egress 100

http://www.logicmonitor.com/support/monitoring/networking-firewalls/netflow/ 

http://www.juniper.net/techpubs/en_US/junos14.1/topics/concept/sflow-ex-series.html

http://www.juniper.net/techpubs/en_US/junos14.1/topics/task/configuration/sflow-ex-series-cli.html 

Next up is piping Netflow into our collector from Juniper's MX routers.  From a LogicMonitor perspective there will be almost no difference.  Unfortunately the degree of difficulty is substantially higher in terms of Juniper configuration.  I'll put that into a separate post; check back later if you are interested.

  • OK, I've finally had a chance to validate this configuration and I can tell you that it works, with a few minor alterations....see below.

    I have deployed this on an MX-80 running Junos 13.3R9.13.  One other relevant addendum to my original "you need to know your MX HW & SW in detail" caveat: I have 20 x1 GE and 2 x 10GE MIC-3D powering my physical interfaces; if you have anything else consult Juniper documentation for sampling support information.  good luck with that :(

    set chassis fpc 1 sampling-instance NETFLOW-INSTANCE

    #####The above statement is valid for MX-240, MX-480, and MX-960 HW, though you will need to specify the fpc you want to use.  Also, there are very likely some limitations with regards to the number of sampling instances per fpc that you can create, the total number of instances that can be configured per chassis, and whether any single instance can span multiple fpc.

    #####The below statement is valid for MX-80 HW.  Given that MX-80 has a single tfeb, there are almost certainly much stricter limitations that govern the configuration of the number and deployment of sampling instances.

    set chassis tfeb0 slot 0 sampling-instance NETFLOW-INSTANCE

     

    #####From here down is the same regardless of MX model, though of course the physical and logical interfaces will vary.


    set chassis network-services ip

    set services flow-monitoring  version9 template LM-V9 option-refresh-rate seconds 25
    set services flow-monitoring  version9 template LM-V9 template-refresh-rate seconds 15
    set services flow-monitoring  version9 template LM-V9 ipv4-template


    set forwarding-options sampling instance NETFLOW-INSTANCE input rate 1 run-length 0
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 port 2055
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 source 192.168.10.1   source-address 192.168.10.1
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 version9 template LM-V9
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output inline-jflow source-address 192.168.10.1

    set interfaces ge-1/3/3 unit 2630 family inet sampling input
    set interfaces ge-1/3/3 unit 2630 family inet sampling output

     

  • I previously posted about sFlow export from Juniper EX/QFX5100 switches.   This time I'll post about exporting Netflow v9 from
    Juniper MX routers.  I'll start by noting that little additional effort is needed as far as LogicMonitor goes and by strongly
    reiterating that you should pay close attention to LM's best practices and think through carefully the details of your
    implementation.
    Then I'll provide these disclaimers: 1)conceptually, this is very similar to the way lots of features
    are implemented --> create a definition and then specify how and where to apply it.  In practice I think it is painfully tedious
    and Juniper's documentation contributes just as much to the difficulty as it does to providing an answer. Expect to burn a few hours to get this working  2)I actually can't
    put this particular method of flow collection into production because it has both HW & SW constraints but I am confident
    that it will work (it mirrors a working IPFIX configuration that was operating with a non-LM collector) as soon as I
    get to the appropriate Juniper SW release (Junos 13.3Rx). The takeaway here is: you need to know your MX HW & SW in detail.
    3)I offer no guarantees, your mileage may vary and ABSOLUTELY use the <commit confirmed> option when you turn this on.

    NOTE also that I haven't included any links to Juniper documentation here, because well, there's too many references and which ones you will need is going to be driven by the MX router HW and SW that you are attempting to implement on.

    set chassis fpc 1 sampling-instance NETFLOW-INSTANCE
    set chassis network-services ip

    set services flow-monitoring  version9 template LM-V9 option-refresh-rate seconds 25
    set services flow-monitoring  version9 template LM-V9 template-refresh-rate seconds 15
    set services flow-monitoring  version9 template LM-V9 ipv4-template


    set forwarding-options sampling instance NETFLOW-INSTANCE input rate 1 run-length 0
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 port 2055
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 source 192.168.10.1
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 version9 template LM-V9
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output inline-jflow source-address 192.168.10.1

    set interfaces ge-1/3/3 unit 2630 family inet sampling input
    set interfaces ge-1/3/3 unit 2630 family inet sampling output

     

  • Hmm...

    The below worked just fine - EX4200 stack running 12.3R6.6

    Unfortunately this is quite busy with NFS and iSCSI traffic so you mostly see that. I should adjust things for different flow samples and perhaps remove the interfaces handling storage for a better view of things.

    I also have this working from Fortigate firewalls which works great as it is all about the Internet and cross-zone traffic and the storage network doesn't flow through.

    protocols {
    ...
        sflow {
            agent-id xxx.xxx.xxx.249;
            polling-interval 20;
            sample-rate {
                ingress 20;
                egress 20;
            }
            source-ip xxx.xxx.xxx.249;
            collector xxx.xxx.xxx.218;
            interfaces ge-0/0/0.0;
            interfaces ge-0/0/1.0;
    ...
            interfaces ge-2/0/22.0;
            interfaces ge-2/0/23.0;
        }
    }

     

  • Hey Mike,

    despite the differences between my posted <set> commands and the <set> commands you issued to generate your config, you have a perfectly valid configuration and your results confirm that.

    What your config and results also demonstrate is that <polling-interval> and <sampling-rate> are variables whose values are not "one size fits all"  so I highly recommend consulting Juniper documentation, experimenting, and then reviewing the results you see in LM to arrive at what works best for you and your needs.

     

  • We don't disagree on anything - just our configurations are very different in implementation but working on both sides.

  • Having issues with version9 template for MX 80 devices. Using the same syntax as previously mentioned for other MX devices, with exception of the chassis tfeb setting and the collectors are having issues not recognizing the template. Using JunOS 13.2R5.10. Any ideas?

    [08-26 21:52:29.189 EDT] [NOTICE] [INFO] [pool-56-thread-1::netflow:752081174] [NetFlowParser$NF9Decoder.decode:978] CAUSE=Don't find template, ACTION=Discard, CONTEXT=deviceId=3209, srcId=32768, templateId=272
    chassis {
        tfeb {
            slot 0 {
                sampling-instance flow-export-01;
            }
        }
    }
    services {
        flow-monitoring {
            version9 {
                template ipv4 {
                    template-refresh-rate {
                        seconds 15;
                    }
                    option-refresh-rate {
                        seconds 25;
                    }
                    ipv4-template;
                }
            }
        }
    }
    forwarding-options {
        sampling {
            instance {
                flow-export-01 {
                    input {
                        rate 1000;
                        run-length 0;
                    }
                    family inet {
                        output {
                            flow-server 10.18.22.174 {
                                port 2055;
                                source-address 10.18.22.1;
                                version9 {
                                    template {
                                        ipv4;
                                    }
                                }
                            }
                            inline-jflow {
                                source-address 10.18.22.1;
                            }
                        }
                    }
                }
            }
        }
    }

     

  • Hey James

    as I said, this takes a while to work through all the moving parts.  I just recently completed an upgrade to JUNOS 13.3Rx and will be attempting this soon and I'm not really looking forward to it.  What I pasted previously was from a working configuration that exported IPFIX from an MX240; I don't have my notes with me but I do recall that IPFIX and v9 were nearly identical procedures.  As you reference, the HW difference between 80 and 240 does come with slight configuration differences.   I expect to post again in the near future once I get it working (or maybe it won't work?), but in the meantime this juniper link may help.  note also that LogicMonitor's netflow configuration documentation page does have specific caveats with regards to v9 and templates. good luck and post details of your results if/when you get the chance.

     https://www.juniper.net/documentation/en_US/junos13.3/topics/task/configuration/inline-flow-monitoring.html

  • OK, I've finally had a chance to validate this configuration and I can tell you that it works, with a few minor alterations....see below.

    I have deployed this on an MX-80 running Junos 13.3R9.13.  One other relevant addendum to my original "you need to know your MX HW & SW in detail" caveat: I have 20 x1 GE and 2 x 10GE MIC-3D powering my physical interfaces; if you have anything else consult Juniper documentation for sampling support information.  good luck with that :(

    set chassis fpc 1 sampling-instance NETFLOW-INSTANCE

    #####The above statement is valid for MX-240, MX-480, and MX-960 HW, though you will need to specify the fpc you want to use.  Also, there are very likely some limitations with regards to the number of sampling instances per fpc that you can create, the total number of instances that can be configured per chassis, and whether any single instance can span multiple fpc.

    #####The below statement is valid for MX-80 HW.  Given that MX-80 has a single tfeb, there are almost certainly much stricter limitations that govern the configuration of the number and deployment of sampling instances.

    set chassis tfeb0 slot 0 sampling-instance NETFLOW-INSTANCE

     

    #####From here down is the same regardless of MX model, though of course the physical and logical interfaces will vary.


    set chassis network-services ip

    set services flow-monitoring  version9 template LM-V9 option-refresh-rate seconds 25
    set services flow-monitoring  version9 template LM-V9 template-refresh-rate seconds 15
    set services flow-monitoring  version9 template LM-V9 ipv4-template


    set forwarding-options sampling instance NETFLOW-INSTANCE input rate 1 run-length 0
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 port 2055
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 source 192.168.10.1   source-address 192.168.10.1
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output flow-server 192.168.1.2 version9 template LM-V9
    set forwarding-options sampling instance NETFLOW-INSTANCE family inet output inline-jflow source-address 192.168.10.1

    set interfaces ge-1/3/3 unit 2630 family inet sampling input
    set interfaces ge-1/3/3 unit 2630 family inet sampling output