create new tag
view all tags

Data Sharing - FMNC

All data gleaned from the core FMNC techniques will be made available via daily downloads which will be available here. The daily downloads are free for anyone to download. We do ask that if you are an established industrial entity that is using the data significant commercial purposes that you consider supporting the FMNC project. Startups with limited budgets and researchers at education institutions are exempt. More information on our current supporting sponsors and how your company can participate can be found here.

The data characteristics are as follows:

  • Format is XML-based. Data is organized on a per-test basis.
  • Locations and clients are anonymized. Locations are securely hashed (source IP basis).
    • User tracking in the public dataset is intentionally removed. Hence, you should be able to infer connections coming from the same IP origination point (ex. NAT point) but will not be able to track Client X across multiple tests.
  • The data contains the test configuration parameters (properly sanitized) and all observed RTT parameters
    • Size of data block, time slice left server, time estimated acknowledgment was received

An example XML is available as a reference point along with parsing code. We are also happy to help you translate it into the same database we use (MongoDB) to help expedite querying and analytics.

The goal of the core public data is to give a rich repository of millions of sample points for researchers to analyze. At the same time, we recognize that this may not be quite what one is looking for from a QoE perspective as individual user variations are lost. Hence, it is possible with appropriate negotiation to get access to per-user tracking (still anonymized but now you can differentiate Client X over time). There are a few different ways to go about doing this:

  • Tweak the request configuration to tag / mark your users (startups or academic researchers)
    • We have a few variants in the works that can be tweaked to track specific users. For the configurable XML variation, a custom tag can be added. The tag would be logged in our database whenever a test is recorded but would not be shared with other users as it would be sanitized prior to public release. A daily query can be made to give you your information or a custom query mechanism could be built (it really is not that custom but does require some plumbing).
    • Alternatively, we can plumb a custom object for request that meets your needs and also gets tagged in the DB. For instance, your client could ask for custom GIFX2382 or some sort of a pattern-based expression.
    • Ping Prof. Striegel for additional details.
  • Help support the FMNC project (gift in-kind, grant support)
    • We can run a variety of aggregated reports that respect privacy-sensitivity and have considerable experience doing this with our NetSense project.
    • Ping us and we would be happy to put equipment or student support to work
  • Download / install your own FMNC install
    • We do plan to look into possibly using Amazon Web Services as an option. This is a longer term goal though so don't wait for us if you want to do this. We think things should work OK with the server as is (emphasis on think).
    • We would love as well if you can contribute code back to the core FMNC project. Ping us for getting Subversion commit access.
  • Work through the IRB process for data sharing
    • For academics, this gives you the richest access to the data. While nearly all of the data is exempt, we do need to follow the proper procedure. For quite a few cases, the public dataset is enough but we can potentially share additional information from the apps that are not easily gleaned otherwise.

To restate the above, if you can get by on the publicly available dataset, use the data as you see fit. It is free now and as long as we can maintain this project / this project being useful. If you need a bit more functionality or just find this tool incredibly useful, we would appreciate it if you can find a way to build in a bit of support (even the smallest amounts are awesome) for the project once you find it useful.

Edit | Attach | Watch | Print version | History: r1 | Backlinks | Raw View | More topic actions
Topic revision: r1 - 2014-06-04 - AaronStriegel
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback