Data Stream Sharing. Richard Kuntschke and Alfons Kemper. Fakultät für Informatik Technische Universität München Germany - PDF

Description
Data Stream Sharing Richard Kuntschke and Alfons Kemper Fakultät für Informatik Technische Universität München Germany 29/03/2006 Data Stream Sharing 1 Overview Introduction and Motivation Subscription

Please download to get full document.

View again

of 28
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information
Category:

Slides

Publish on:

Views: 52 | Pages: 28

Extension: PDF | Download: 0

Share
Transcript
Data Stream Sharing Richard Kuntschke and Alfons Kemper Fakultät für Informatik Technische Universität München Germany 29/03/2006 Data Stream Sharing 1 Overview Introduction and Motivation Subscription Language Data Stream Sharing Conclusion and Outlook 29/03/2006 Data Stream Sharing 2 Overview Introduction and Motivation Subscription Language Data Stream Sharing Conclusion and Outlook 29/03/2006 Data Stream Sharing 3 Preliminaries The StreamGlobe Data Stream Management System (DSMS) (VLDB 2005) Grid-based P2P network Super-Peers Thin-Peers Speaker-Peer XML Data Streams (W)XQuery Subscriptions Query processing with FluX (VLDB 2004) 29/03/2006 Data Stream Sharing 4 29/03/2006 Data Stream Sharing 5 Goals and Challenges Optimize incrementally registered subscriptions Find suitable input data in the network Reduce network traffic and peer load 29/03/2006 Data Stream Sharing 6 Optimization Techniques Data Stream Sharing In-network query processing Multi-subscription optimization Treat queries and data streams symmetrically Cost-based optimizer 29/03/2006 Data Stream Sharing 7 Motivation photons SP 4 SP 6 photons SP 4 SP 6 P 0 Query 4 P 0 Query 4 SP 0 SP 2 P 4 SP 0 SP 2 P 4 Query 1 Super-Peer Backbone Query 2 Query 1 Super-Peer Backbone Query 2 SP 5 SP 7 P 2 SP 5 SP 7 P 2 P 1 Query 3 P 1 Query 3 SP 1 SP 3 P 3 SP 1 SP 3 P 3 Without Data Stream Sharing With Data Stream Sharing 29/03/2006 Data Stream Sharing 8 Overview Introduction and Motivation Subscription Language Data Stream Sharing Conclusion and Outlook 29/03/2006 Data Stream Sharing 9 Example Data Stream photons photon* coord phc en det_time cel det ra dec dx dy 29/03/2006 Data Stream Sharing 10 Example Data Vela Supernova Remnant RXJ Supernova Remnant 29/03/2006 Data Stream Sharing 11 WXQuery (Windowed XQuery) photons for $p in stream( photons )/photons/photon where $p/coord/cel/ra = and $p/coord/cel/ra = and $p/coord/cel/dec = and $p/coord/cel/dec = return vela {$p/coord/cel/ra} {$p/coord/cel/dec} {$p/phc} {$p/en} {$p/det_time} /vela /photons 29/03/2006 Data Stream Sharing 12 WXQuery (Windowed XQuery) photons for $w in stream( photons )/photons/photon [coord/cel/ra = and coord/cel/ra = and coord/cel/dec = and coord/cel/dec = -40.0] /photon/det_time diff 20 step 10 let $a := avg($w/photon/en) return avg_en {$a} /avg_en /photons 29/03/2006 Data Stream Sharing 13 Data Windows /photon/det_time diff 20 step det_time count 4 step det_time 29/03/2006 Data Stream Sharing 14 Overview Introduction and Motivation Subscription Language Data Stream Sharing Conclusion and Outlook 29/03/2006 Data Stream Sharing 15 Query 1 photons for $p in stream( photons )/photons/photon where $p/coord/cel/ra = and $p/coord/cel/ra = and $p/coord/cel/dec = and $p/coord/cel/dec = return vela {$p/coord/cel/ra} {$p/coord/cel/dec} {$p/phc} {$p/en} {$p/det_time} /vela /photons 29/03/2006 Data Stream Sharing 16 Abstract Properties of Query 1 29/03/2006 Data Stream Sharing 17 Query 2 photons for $w in stream( photons )/photons/photon [coord/cel/ra = and coord/cel/ra = and coord/cel/dec = and coord/cel/dec = -40.0] /photon/det_time diff 20 step 10 let $a := avg($w/photon/en) return avg_en {$a} /avg_en /photons 29/03/2006 Data Stream Sharing 18 Abstract Properties of Query 2 29/03/2006 Data Stream Sharing 19 Data Stream Discovery and Cost Model Data Stream Discovery Start at origin of referenced stream Search forward (BFS or DFS) in the network graph Pruning Cost Model Parameters Network traffic Computational load on peers 29/03/2006 Data Stream Sharing 20 Data Stream Discovery Example photons 4 6 P 0 Query 4 Query 1 P Super-Peer Backbone 5 P 4 Query 2 P 2 7 Query P 3 29/03/2006 Data Stream Sharing 21 Window-based Aggregation det_time Query 3: /photon/det_time diff 20 step 10 Query 4: /photon/det_time diff 60 step mod 20 = 0 40 mod 10 = 0 20 mod 10 = 0 60 div 20 = 3 40 div 10 = 4 20 div 10 = 2 29/03/2006 Data Stream Sharing 22 Performance Evaluation Preliminaries 4 x 4 Grid Topology 16 Super-Peers 2 Data Streams Real astrophysical data photons data streams 100 Queries Randomly generated Query Templates for Selection/Projection/Aggregation queries Constant values for selection predicates and data window definitions randomly chosen from predefined set 29/03/2006 Data Stream Sharing 23 Performance Evaluation Peer Load Data Shipping Query Shipping Stream Sharing Average CPU Load (%) SP0 SP2 SP4 SP6 SP8 SP10 SP12 SP14 Peers 29/03/2006 Data Stream Sharing 24 Performance Evaluation Network Traffic 200 Data Shipping Query Shipping Stream Sharing Network Traffic (MBit) SP0 SP2 SP4 SP6 SP8 SP10 SP12 SP14 Peers 29/03/2006 Data Stream Sharing 25 Overview Introduction and Motivation Subscription Language Data Stream Sharing Conclusion and Outlook 29/03/2006 Data Stream Sharing 26 Conclusion What has been presented: Subscription language Properties approach Cost model Algorithms for data stream sharing Data Stream Sharing takes three steps: Properties construction Identification of shareable streams through properties matching Plan generation, installation, and execution 29/03/2006 Data Stream Sharing 27 Outlook Advanced Data Stream Sharing Improved properties structure Support for nested queries Data stream widening Dynamic optimizer Scalability Hierarchical network organization Fully distributed network organization 29/03/2006 Data Stream Sharing 28
Related Search
Similar documents
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks