StreamGlobe: Adaptive Anfragebearbeitung und Optimierung auf Datenströmen - PDF

Description
StreamGlobe: Adaptive Anfragebearbeitung und Optimierung auf Datenströmen B. Stegmaier und R. Kuntschke TU München Fakultät für Informatik Lehrstuhl III: Datenbanksysteme

Please download to get full document.

View again

of 19
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Information
Category:

Music & Video

Publish on:

Views: 17 | Pages: 19

Extension: PDF | Download: 0

Share
Transcript
StreamGlobe: Adaptive Anfragebearbeitung und Optimierung auf Datenströmen B. Stegmaier und R. Kuntschke TU München Fakultät für Informatik Lehrstuhl III: Datenbanksysteme Outline Motivation StreamGlobe The StreamGlobe Approach Architecture Overview Current and Future Research Conclusion 9. Mai 2006 StreamGlobe 2 Exemplary Initial Situation B A WLAN b Network Consists of peers Given or grown topology Data Sources Provide XML data stream Possibly infinite streams (e.g., sensor measurements) User requests Continuous queries Query language XQuery Registered at a peer 9. Mai 2006 StreamGlobe 3 General Traditional Approach A B 1. Register requests 2. Lookup data sources 3. Establish data transfer Peers may connect arbitrarily 4. Process / Execute requests 5. Routing of streams b 9. Mai 2006 StreamGlobe 4 General Traditional Approach (ctd.) A 2 B Drawbacks 1. Transmission of useless data 2. Redundant transmissions 3. Multiple request evaluation 3 1 Network congestion and processing overhead 3 b 9. Mai 2006 StreamGlobe 5 Why StreamGlobe? Other Systems / previous work E.g. Cougar, TelegraphCQ, Multicast techniques: Focus on specific aspects (e.g., query optimization) Tailored to specific domains StreamGlobe Contribution is combination of techniques: In-network query processing combined with routing Constitutes a generic infrastructure Independent of domain Efficient data stream transformation and distribution 9. Mai 2006 StreamGlobe 6 Outline Motivation StreamGlobe The StreamGlobe Approach Architecture Overview Current and Future Research Conclusion 9. Mai 2006 StreamGlobe 7 The StreamGlobe Approach A a B Overload ab Filter ab Intelligent Routing Push query execution into network Multicast routing techniques Data Stream Clustering Multi-query optimization ab Reduce network traffic Reduce processing cost Load balancing b 9. Mai 2006 StreamGlobe 8 Basic Concepts P2P Network Topology No arbitrary communication Communication via transfer paths No fixed P2P topology Classification of peers Thin-Peers Super-Peers Constitution of a super-peer backbone Hierarchical organization Speaker-peer responsible for certain subnet 9. Mai 2006 StreamGlobe 9 StreamGlobe Peer Architecture XQuery Subscriptions register StreamGlobe Interface Optimization Query Engine Globus Toolkit XML Data Streams Metadata Management Based upon Open Grid Services Architecture (OGSA) Integration similar to OGSA- DAI or OGSA-DQP Layers as grid-services Availability according to peer capabilities Message exchange via RPC and notifications Data stream transfer via direct TCP connections 9. Mai 2006 StreamGlobe 10 StreamGlobe Interface Registration of XML data streams Possibilities As individual data stream As part of a virtual data stream Specification of schema with XML Schema Registration of subscription rules At a special peer (user device) Specified using XQuery 9. Mai 2006 StreamGlobe 11 Metadata Management Managed information Network topology Registered subscriptions and data streams Statistics of data streams Based upon service-data elements of Globus Toolkit Maintenance E.g. peers joining/leaving, change of subscriptions, etc. Using notification mechanisms of Globus Toolkit 9. Mai 2006 StreamGlobe 12 Optimization Goals 1. Register arbitrary subscriptions at any peer 2. Reduce network traffic 3. Optimize evaluation of many subscriptions Achievement Pushing query execution into the network (1), (2) and (3) Multi-query optimization (3) Early filtering of data streams resp. evaluation of subscriptions (2) Data stream clustering (2) 9. Mai 2006 StreamGlobe 13 Multi-Query Optimization b Performed by speaker-peer Analyze subscriptions and streams Common subqueries Query a Filter a Filter b Query ab Re-usability of streams Based on properties of subscriptions / streams Computes Filters and queries Data stream clustering Execution locations 9. Mai 2006 StreamGlobe 14 Query Execution Basic concepts Streaming evaluation and push-based techniques Preclude unbounded buffering by requiring window constraints Extensibility by means of mobile code Evaluation of subscriptions with FluX Designed for streaming processing of XQuery Event-based extension to XQuery Usage of schema information for buffer minimization 9. Mai 2006 StreamGlobe 15 Outline Motivation StreamGlobe The StreamGlobe Approach Architecture Overview Current and Future Research Conclusion 9. Mai 2006 StreamGlobe 16 Current and Future Research Current Research Optimization techniques Extension of FluX Future Research Quality-of-Service management Explicit load balancing Load shedding techniques Construction of overlay network 9. Mai 2006 StreamGlobe 17 Conclusion StreamGlobe Exploiting in-network query processing capabilities In combination with data stream clustering Minimization of network traffic Query execution with FluX Efficient and scalable execution of subscriptions Multi-query optimization Parallelization and load balancing in the network 9. Mai 2006 StreamGlobe 18 Related Work Aberer, Cudré-Mauroux, Datta, Despotovic, Hauswirth, Punceva, Schmidt. P-Grid: a self-organizing structured P2P system. SIGMOD Record 32(3), 2003 Braumandl, Keidl, Kemper, Kossmann, Kreutz, Seltzsam, Stocker. ObjectGlobe: Ubiquitous query processing on the Internet. VLDBJ 2001 Braumandl, Kemper, Kossmann. Quality of Service in an Information Economy. TOIT 2003 Chandrasekaran, Cooper, Deshpande, Franklin, Hellerstein, Hong, Krishnamurthy, Madden, Raman, Reiss, Shah. TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. CIDR 2003 Keidl, Kreutz, Kemper, Kossmann. A Publish & Subscribe Architecture for Distributed Metadata Management. ICDE 2002 Krämer, Seeger. PIPES A Public Infrastructure for Processing and Exploring Streams. SIGMOD 2004 Löser, Siberski, Wolpers, Nejdl. Information Integration in Schema-Based Peer- To-Peer Networks. CAiSE 2003 Yang, Garcia-Molina. Designing a Super-Peer Network. ICDE 2003 Yao, Gehrke. The Cougar Approach to In-Network Query Processing in Sensor Networks. SIGMOD Record 31(3), Mai 2006 StreamGlobe 19
Related Search
Similar documents
View more...
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks