UNIVERSITY OF TORONTO

Fighting back:  Using observability tools to improve  the DBMS (not just diagnose it) Ryan Johnson

UNIVERSITY OF

TORONTO

UNIVERSITY OF TORONTO

DBMS/OS interface is tricky terrain Deadlocks Correctness Scheduling

Races Performance

Resource  management

Locality

UNIVERSITY OF TORONTO

Fine‐grained, targeted observation? Image credit: Müllner, Bodner, Mannor. BJO 83:8, 1999. 

.5mm

+ Image credit: Johns Hopkins Dept. of Orthopedic Surgery

Real goal: fine‐grained, targeted fixes

UNIVERSITY OF TORONTO

DTrace: making systems transparent Instrument majority of OS and all user‐level functions

Trace events  generated by   domain‐specific  “providers”

Domain‐specific  language specifies  probe actions

Zero disabled  probe cost

Safe: designed for  non‐wizard use with  production systems

Two‐way OS/app  communication via  process memory

UNIVERSITY OF TORONTO

Disclaimer This talk is about a new way of looking at our  tools, not about any one (use of a) tool. For concreteness, we’ll discuss a specific tool  (DTrace) and problem class (scheduling) that I  happen to be familiar with.  Hopefully these examples inspire other/better  uses for active observers within the DBMS. 

UNIVERSITY OF TORONTO

Example: diagnosing contention Sample‐based profiling  (OS service) Post‐processing  identifies samples  due to contention (lock  waits, atomic ops, etc.) Work breakdown 100%

Contention Compute More client threads

UNIVERSITY OF TORONTO

Spinning and thread preemption

!

!

DBMS records lock acquire/release, DTrace does the rest

7

UNIVERSITY OF TORONTO

Priority inversion unmasked DTrace tracks locks held  by pre‐empted threads,  samples call stacks Worker threads record  held/attempted locks  in thread‐local storage

DTrace isolates samples  where threads wait on  pre‐empted locks

Work breakdown 100%

Prio‐invert Contention Compute

Post‐processing  differentiates samples  due to contention and  priority inversion

More client threads

Neither OS nor DBMS can achieve this alone

UNIVERSITY OF TORONTO

Example: Queuing and pre‐emption

Previous latch holder (deallocated) W=0 N=

W=0 N=

Current latch holder

tail= W=1 N=

W=1 N=

W=1 N=

W=1 N=

UNIVERSITY OF TORONTO

Example: Queuing and pre‐emption

tail=

Preempted! W=0 N= Current latch holder

W=1 N=

W=1 N=

W=1 N=

W=1 N=

UNIVERSITY OF TORONTO

Preemption‐resistant queuing New latch holder

W=1 N=

W=0 N= Preempted! W=0 N=

W=1 N=

W=1 N= W=1 N=

tail= W=1 N=

W=1 N=

Current latch holder

DTrace dissolves queue, wakeful threads simply rejoin

UNIVERSITY OF TORONTO

Conclusions • Cooperation between DBMS and observability tools can be a powerful thing • Recurring theme: two‐way communication  between OS/DBMS • What other active uses might be out there? • How to adapt our tools for these new uses?

Fighting back: Using observability tools to improve the ...

Using observability tools to improve the DBMS (not just ... For concreteness, we'll discuss a specific tool. (DTrace) and ... Sample-based profiling. (OS service).

605KB Sizes 5 Downloads 241 Views

Recommend Documents

Using Meta-Reasoning to Improve the Performance of ...
CCL, Cognitive Computing Lab. Georgia Institute of ..... Once a game finishes, an abstracted trace is created from the execution trace that Darmok generates.

Using The Simpsons to Improve Economic Instruction ...
students the opportunity to practice the economic analysis of public policy issues. Empirical research on the .... prohibition seen in Springfield and the narcotics market in the United States are clear. Showing this ..... While we did not collect co

Using the contextual redefinition strategy to improve ... - PUCV Inglés
The whole class sat the test and the score average was 34 (see Appendix E: Vocabulary Size Test. Scores), which ..... Retrieved from http://ejournal.upi.edu/index.php/L-E/article/view/583 ... http://181.112.224.103/bitstream/27000/3081/1/T-UTC-4018.p

Using the contextual redefinition strategy to improve ... - PUCV Inglés
students, and then the way in which the English subject is addressed in the school. .... observe those characteristics in the 10th graders I am teaching when I wanted to introduce a simple ..... Next, the students will have to be able to analyse the

Using Machine Learning to Improve the Email ... - Research at Google
Using Machine Learning to Improve the Email Experience ... services including email, and machine learning has come of ... Smart Reply: Automated response.

Using Meta-Reasoning to Improve the Performance of Case-Based ...
formance over a period of time. Fox and Leake [8] developed a system to improve the retrieval of a CBR system using meta-reasoning. Their work used a model.

101 Ways to Improve Customer Service Training, Tools ...
and job aids and are frequently supplemented with CD-ROMs, websites, and ... transmission, electronic reproduction or inclusion in any publications offered for sale or used for com- .... Getting Down to Business: Customer Comment Cards.

Using Task Load Tracking to Improve Kernel Scheduler Load ...
Using Task Load Tracking to Improve Kernel Scheduler Load Balancing.pdf. Using Task Load Tracking to Improve Kernel Scheduler Load Balancing.pdf. Open.

Using Data to Improve Student Achievement
Aug 3, 2008 - Data are psychometrically sound, such as reliable, valid predictors of future student achievement, and are an accurate measure of change over time. • Data are aligned with valued academic outcomes, like grade-level out- come standards

Using targeted feedback surveys to inform and improve ...
Many Koreans are unused to CLT as the Korean education system promotes rote learning, memorisation .... Asian EFL Journal 4 (2), [Online]. Available from: ...

Using Argument Mapping to Improve Critical ... - Semantic Scholar
Feb 4, 2015 - The centrality of critical thinking (CT) as a goal of higher education is uncon- troversial. In a recent high-profile book, ... dents college education appears to be failing completely in this regard: “With a large sample of more than

Using Relaxations to Improve Search in Distributed ...
Computer Science, University College Cork, Ireland. Abstract. Densely ..... Autonomous Agents and Multi-Agent Systems 3(2) (2000) 185–207. 4. Modi, P., Shen ...

How Windows is using hardware to improve security - BlueHat IL
Terminate process if invalid target. Indirect. Call. Kernel Control Flow Guard improves protection against control flow hijacking for kernel code. Paired with HVCI to ensure both code integrity and control flow integrity. OSR REDTEAM targeted kCFG bi

Using a Sensitivity Measure to Improve Training ...
Engineering, Hohai University, Nanjing 210098, China (email: [email protected]). In our study, a new learning algorithm based on the MRII algorithm is developed. We introduce a sensitivity of. Adalines, which is defined as the probability of an Adalin

Using Search-Logs to Improve Query Tagging - Slav Petrov
Jul 8, 2012 - matching the URL domain name is usually a proper noun. ..... Linguistics, pages 497–504, Sydney, Australia, July. Association for ...