Tags:
create new tag
view all tags
Why do we publish our reviews?

Paper C X. Hu, L. Meng, A. Striegel, "Evaluating the Raw Potential for Device-to-Device Caching via Co-Location," in Proc. of MobiSPC 2014, Niagara Falls, Canada, August 2014. Review

The reviews are included unmodified aside from added carriage returns (to render better via verbatim rendering in TWiki) and section headers to easily delineate each of the reviews. The submitted PDF is also included to provide better context for the reviewers comments.

Review 1 (Weak Reject)

PAPER: 30
TITLE: Evaluating the Raw Potential for Device-to-Device Caching via Co-Location
AUTHORS: Xueheng Hu, Lei Meng and Aaron Striegel

OVERALL EVALUATION: -1 (weak reject)
Relevance w.r.t. CFP: 5 (Highly relevant)
Theoretical and Practical Significance: 4 (Significant)
Technical Quality and Accuracy: 1 (Flawed with major errors)
Quality of Presentation: 5 (excellent)
Best paper award: 1 (I object against an award for this paper)
If accepted, downgrade? (full paper --> short paper): 4 (Full paper)

----------- REVIEW -----------
In this paper, the authors present a feasibility study on device-to-device caching when exploiting geolocality. 
The authors use data from 200 Nexus S phones distributed to undergraduates to study how much data these 
user download over time, and how often these users are within Bluetooth range of each other. The author's 
model how data could propagate among these users if the mobile devices cached data and exchanged it 
with peers when they are within range.

The manuscript is very polished, and easy to read. The figures are clear, and the metrics used seem 
appropriate for the given study. However, the authors position this study as a test-case to see if 
device-to-device caching is feasible, but their results seem overly optimistic due to the relaxed set of 
assumptions that are used.

First, the author's model considers the size of data blocks and signal strength, but not the length of 
contact between mobile devices. Thus, the author's model appears to greatly overestimate the potential 
of sharing by assuming all cached blocks can be exchanged between devices when they are in range. 
Also, the author's model doesn't appear to take into account the point-to-point nature of Bluetooth, i.e. 
if my device is the range of 10 people, I need to transfer data to/from each partner serially, thus potentially 
reducing sharing opportunities.

Second, the author's study is (mostly) oblivious to the type of content downloaded by devices. The author's 
do study the impact of content-age on caching, which is a useful starting point. But there are many additional 
issues, like: separating public content from private content, distinguishing data from separate apps that 
different users may or may not have installed, etc. It is disingenuous to claim that devices could exchange 
hundreds of megabytes of data without also quantifying how much of that data is useful, i.e. results in cache hits.

This overestimation problem is exemplified in the author's results. The authors show that each device is 
caching around 4GB of content per day in their simulations. While it is feasible for a modern mobile device 
to store this amount of data, it seams highly unlikely that this data is useful. Consider a recent report which 
measured the monthly data consumption of mobile devices in the US at 1.3GB per month 
(https://gigaom.com/2014/02/05/cisco-the-u-s-is-officially-in-the-gigabyte-era-of-mobile-data-consumption/) 
If each user is caching 4GB of content each day for 30 days, and 1.3GB are useful per month, that results 
in a cache hit rate of 1% (assuming a perfect hit rate).

Review 2 (Reject)

----------------------- REVIEW 2 ---------------------
PAPER: 30
TITLE: Evaluating the Raw Potential for Device-to-Device Caching via Co-Location
AUTHORS: Xueheng Hu, Lei Meng and Aaron Striegel

OVERALL EVALUATION: -2 (reject)
Relevance w.r.t. CFP: 4 (Relevant)
Theoretical and Practical Significance: 3 (Reasonable)
Technical Quality and Accuracy: 3 (Reasonable)
Quality of Presentation: 4 (good)
Best paper award: 1 (I object against an award for this paper)
If accepted, downgrade? (full paper --> short paper): 3 (Short paper)

----------- REVIEW -----------
This paper studies the feasibility of caching data on mobile devices while they are in physical proximity. 
For this, data are collected and analyzed via a mobile platform called NetSense, with nearly 200 
undergraduate students in the University of Notre Dame. The collocation of the mobile users is 
established via Bluetooth discovery. The actual exchange of data is performed via either cellular or 
wifi interface.

The study concludes that it is feasible to perform caching of data and sharing between collocated 
devices since the actual storage space needed is not that great for nowadays mobile devices 
(3-4GB per device) and the diurnal and other behavioral patterns of mobile users allows such 
exchanges to happen.

However, the study does not address a number of important issues which arise from this setup and 
also should provide more details on the platform, the simulator used and the data at hand to aid 
understanding and how the lessons were extracted.

Questions and comments on content

In the simulation study, no limits were imposed (storage space, rate of data exchanged, etc), to study 
the best possible scenario for the caching option. However, it would have been very useful to also 
present results from simulations that use the NetSense mobility of users coupled with some realistic 
values on parameters such as:
*) rate of data exchanged,
*) overhead for establishing a connection between devices,
*) impact on mobile battery while using the phone interfaces for the data transfers and caching

How was the simulator validated to produce trustworthy results?

How would the results change if the simulator used was a widely accessible one like Network Simulator 
(NS) which would take into account the mobility of nodes as extracted from the NetSense traces and 
perform realistic exchanges of data via simulation of interfaces, etc.?

Indeed, the "Propagation Volume" is an intuitive metric defined to help the presentation of some results. 
How is that a contribution, though?

A lot of details are missing from Section 3 on the NetSense data. In particular, it would be expected to 
report more details on the characteristics of the mobility of users, online and offline times, data exchanged 
via interfaces etc., and how all these characteristics affect the simulation of data caching.

The definitions presented in the beginning of Section 4 should be revisited. There are various issues 
with how they are phrased and how the particular quantities are connected with time. For example, 
does C_mni represent the blocks downloaded from other nodes, or generated? The definition of 
C_mni,mnj implies "generated". Also, where is CL_i,j used in the rest of the paper?

It is not clear from the paragraphs before Section 4.1 that the simulation time is a day. Instead, someone 
has to assume that from the plots presented later.

Is is not clear how the Propagation Volume "improves the earlier metrics". Furthermore, its definition in 
expression (2) should be better stated, using summation notation instead of "total_volume" which is unclear.

The explanation of the experimental results can be improved to provide more details on the following observations:
*) in figure 1 the RSSI line for >= 65dBm shows that there is no change of the propagation ratio until 8-9am. 
How does that relate with the mobility of users? Similarly for the other lines as well.
*) in figure 2 there's an increase of the propagation delay between 12-4pm. How is that explained? 
Does this contradict the mobility expected during those times in the university area?
*) it would be helpful to see how the diurnal behavior works on a weekly schedule of users vs. the metrics shown, 
instead of just on a day.
*) In figures 3, 5, 6 and 7 there is a "valley" between 12-4pm and two peaks between 8-12 and 4-8pm. How does 
this connect with the mobility patterns of users?
*) the results shown in figure 8 should be discussed in text in more detail.

Comments on presentation

Section 1:
"replete" -> "complete"?

Section 2:
Citations with more than one author need "et al.", not only the first author's name.

Section 4:
The penultimate sentence of 4.1 should be reworded.
The first sentence of 4.2.3 could be reworded. Also the last sentence of the same paragraph ("... at the noon is 350MB").

Review 3 (Accept)

----------------------- REVIEW 3 ---------------------
PAPER: 30
TITLE: Evaluating the Raw Potential for Device-to-Device Caching via Co-Location
AUTHORS: Xueheng Hu, Lei Meng and Aaron Striegel

OVERALL EVALUATION: 2 (accept)
Relevance w.r.t. CFP: 4 (Relevant)
Theoretical and Practical Significance: 3 (Reasonable)
Technical Quality and Accuracy: 3 (Reasonable)
Quality of Presentation: 3 (Room for improvement, but readable)
Best paper award: 2 (I hold no strong views)
If accepted, downgrade? (full paper --> short paper): 4 (Full paper)

----------- REVIEW -----------
This paper focus on an interesting topic, the device-to-device caching via opportunistic networking. 
They collect and evaluate the real-world data from about 200 students through their NetSense. To evaluate 
the potential for proximity-based caching, they define propagation ratio, propagation latency and propagation 
volume. The result of the analysis shows D2D caching could achieve good performance potentially. 

In the evaluation of propagation ratio, the author use RSSI as an indicator of channel condition.
If the RSSI value is smaller than a threshold(-80dbm), they can exchange cache. However, usually,
the co-location period is not long enough to share blocks of caching data(how large is the block size?). Moreover, 
during the time period, 1 hour in the paper, the channel condition may change which result in loss of data. Another 
concern is that, considering the amount of data exchange indicates by storage requirement, the power consumption 
could be an issue.

Review 4 (Weak Accept)

----------------------- REVIEW 4 ---------------------
PAPER: 30
TITLE: Evaluating the Raw Potential for Device-to-Device Caching via Co-Location
AUTHORS: Xueheng Hu, Lei Meng and Aaron Striegel

OVERALL EVALUATION: 1 (weak accept)
Relevance w.r.t. CFP: 4 (Relevant)
Theoretical and Practical Significance: 4 (Significant)
Technical Quality and Accuracy: 4 (good)
Quality of Presentation: 4 (good)
Best paper award: 2 (I hold no strong views)
If accepted, downgrade? (full paper --> short paper): 4 (Full paper)

----------- REVIEW -----------
This paper analyzed the potential of pre-staging of the popular contents between smartphone 
users for three key features: co-location, traffic volume, and device state. The authors found 
that the propagation ratios in terms of both file count and volume demonstrated that there exist 
considerable potential for D2D content caching. Therefore, the authors posited this particular 
domain worth more attention from the research community.

This paper is well written and easy to understand. Through extensive experiments, the authors 
illustrated that the pre-stage device-to-device content caching is worth for the smartphone networks.

Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf MobiSPC_30_Hu.pdf r1 manage 200.9 K 2014-06-20 - 01:45 XuehengHu Camera-ready version
Edit | Attach | Watch | Print version | History: r3 < r2 < r1 | Backlinks | Raw View | More topic actions
Topic revision: r3 - 2014-06-20 - XuehengHu
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback