Starting with the premise that "Performance is a Feature", Matt Warren will show you how to measure, what to measure and how to get the best performance from your .NET code.
We will look at real-world examples from the Roslyn code-base and StackOverflow (the product), including how the .NET Garbage Collector needs to be tamed!
The presentation covers:
Why we should care about performance
Pitfalls to avoid when measuring performance
How the .NET Garbage Collector can hurt performance
Real-world performance lessons from open-source code
The webinar recording can be found here: http://www.postsharp.net/blog/post/webinar-recording-performance-is-a-feature
Hidden Gems of Performance Tuning: Hierarchical Profiler and DML Trigger Opti...
In any large ecosystem, there are always areas that stay in the twilight, outside of the public’s attention. This deep dive attempts to change the trend regarding two, at first glance, unrelated PL/SQL topics: hierarchical profiler (HProf) and database triggers. But if you look closer, there’s something in common: they’re significantly underused! HProf because nobody heard about it, database triggers because of decades-old stigma. Let’s put both of them back into our development toolset!
Part #1. One of the most critical FREE SQL and PL/SQL performance tuning tools is almost totally unknown! If you ask, how much time is spent on routine A? How often is function B called? Most developers would hand-code something instead of using the Oracle PL/SQL HProf. This isn’t because the provided functionality is disliked, but because developers aren’t aware of its existence! This presentation is an attempt to alter this trend and reintroduce HProf to a wider audience.
Part #2. There isn’t anything “evil” about database triggers; they just have to be used where they can actually solve problems. In this presentation, various kinds of triggers will be examined from a global system optimization view, including tradeoffs between multiple goals (e.g., depending upon the available hardware, developers can select either CPU-intensive or I/O-intensive solutions). This presentation will focus on the most common performance problems related to different kinds of DML triggers and the proper ways of resolving them.
Data Tracking: On the Hunt for Information about Your Database
Behind the scenes, Oracle databases hide a myriad of processes to ensure that your data can be safely stored and retrieved. These processes also leave “tracks” (or they COULD leave tracks if you set them up properly). These tracks, together with application-specific data, create a complete representation of the system’s day-to-day activity. Too often this representation is lost at the DBA/Developer borderline, mostly because one side is not aware of the needs of the other. This presentation strives to bridge this gap. It focuses on key sources of database information and techniques that are useful for both DBAs and developers:
- Data Dictionary
- Oracle Logging
- Oracle Tracing
- Advanced code instrumentation
This document discusses techniques for detecting and preventing SQL injection using the Percona Toolkit and Noinject!. It begins by introducing SQL injection and how attackers can modify SQL queries without changing server code. It then discusses using query fingerprints to detect new queries that may indicate injection attempts. The Percona Toolkit tools pt-query-digest and pt-fingerprint are used to generate and store fingerprints in a whitelist. Pt-query-digest can detect new fingerprints that have not been reviewed. The Noinject! proxy script uses fingerprints to inspect queries in real-time and block any that do not match whitelisted patterns. The document concludes by discussing limitations and ways to improve the fingerprinting approach.
The document discusses optimization of Real Application Clusters (RAC) in Oracle 12c. It provides background on the author and outlines common root causes of RAC performance issues such as CPU/memory starvation, network issues, and excessive dynamic remastering. The document then presents golden rules for RAC diagnostics including avoiding focusing only on top wait events, eliminating infrastructure issues, identifying problem instances, examining both send and receive side metrics, and using histograms. Specific techniques are described for analyzing wait events like gc buffer busy.
Flexviews is a materialized view solution for MySQL. This set of slides introduces Flexviews concepts, and gives some examples in how to use it and what to use it for.
This presentation is the attempt to switch sides and show code management from the developer's point of view. It stays outside of various VCS solutions and focuses on hands-on approaches: activity control via system triggers, conditional compilation, synonym manipulation, utilization of Edition-Based Redefinition (EBR).
Percona Live (https://www.percona.com/live/data-performance-conference-2016/sessions/performance-schema-mysql-troubleshooting)
The performance schema in MySQL version 5.6, released in February, 2013, is a very powerful tool that can help DBAs discover why even the trickiest performance issues occur. Version 5.7 introduces even more instruments and tables. And while all these give you great power, you can get stuck choosing which instrument to use.
In this session, I will start with a description of a typical problem, then guide you how to use the performance schema to find out what causes the issue, the reason for unwanted behavior and how the received information can help you solve a particular problem.
Traditionally, performance schema sessions teach what is in contained in tables. I will, in contrast, start from a performance issue, then demonstrate which instruments and tables can help solve it. We will discuss how to setup the performance schema so that it has minimal impact on your server.
The document discusses tracing parallel execution in Oracle. It provides an example query using parallel execution hints and describes how to enable and generate trace files for the query coordinator process and parallel execution slave processes. It then summarizes the key events shown in the trace files, such as setting up parallel execution, full table scans by slave processes, and consolidating results.
QA your code: The new Unity Test Framework – Unite Copenhagen 2019
Are you involved in testing or QA on projects in Unity? In these slides, you'll get an overview of the state of Unity for all things testing-related, and have the opportunity to share your stories of success, failure, pain, and glory. Learn from your fellow developers and give feedback on how Unity could help you hold your projects to a higher standard of quality. You will also get an introduction to the newest features in the Test Framework.
Speakers:
Christian Warnecke - Unity
Richard Fine - Unity
Watch the session on YouTube: https://youtu.be/wTiF2D0_vKA
The document discusses how the Oracle optimizer can sometimes choose suboptimal execution plans, leading to performance deterioration. It presents a scenario where the same query runs much slower when bind variables are used. The document then shows how SQL profiles can be used to enforce a better execution plan. It argues that manually creating profiles is not ideal for 24/7 environments. The document proposes using machine learning for outlier detection to identify performance issues and then automatically generate SQL profiles to address the issues. Code examples are provided for outlier detection and generating profiles through the Oracle API to allow automating the process.
A close encounter_with_real_world_and_odd_perf_issues
This document discusses a performance issue where a database experienced high CPU usage in the kernel mode. Tracing tools identified that detaching from multiple shared memory segments during connection release was causing the high CPU. The database server had a NUMA architecture, causing the database instance to create multiple shared memory segments across NUMA nodes. Increasing the shared memory size limit did not resolve it, as the instance was optimizing for NUMA.
UKOUG version of a presentation trying to establish the sensible limits of parallelism on a couple of hardware configurations. Detailed white paper is at http://oracledoug.com/px_slaves.pdf
This document discusses SCAN, VIP, and HAIP in Oracle RAC environments. It provides details on:
- VIP - a virtual IP address that is not statically linked to a single node, allowing for faster failovers. Each node has a VIP.
- SCAN - a single virtual IP and listener that provides load balancing and high availability. SCAN acts as an abstraction layer so client connect strings do not need to change.
- HAIP - high availability IP addresses that allow clusterware and the database to use plumbed IP addresses for private interconnect traffic via solutions like bonding and trunking.
This document summarizes several myths about database redo, undo, commit, and rollback operations. It presents test cases and analysis to debunk the myths. The author is an experienced Oracle DBA who specializes in performance tuning and internals. Sample redo records are displayed and analyzed to explain how operations like rollback do generate redo. The document aims to clarify misunderstandings about the internal workings of Oracle's transaction and redo logging.
Database & Technology 1 _ Tom Kyte _ Efficient PL SQL - Why and How to Use.pdf
Thomas Kyte discusses effective techniques for writing PL/SQL code. Some key points:
1) Use PL/SQL for data manipulation as it is tightly coupled with SQL and most efficient.
2) Write as little code as possible by leveraging SQL and thinking in sets rather than loops.
3) Use static SQL where possible for compile-time checking and dependency tracking. Dynamic SQL should only be used when static SQL is impractical.
4) Leverage packages to reduce dependencies, increase modularity, and support overloading and encapsulation.
5) Employ bulk processing techniques like bulk collects to minimize round trips to the database.
deathstar is a tool for seamlessly running R code across local and cloud computing nodes. It uses ZeroMQ to distribute data and code across nodes. The deathstar daemon runs on each node and listens for work. The R package exposes a zmq.cluster.lapply function similar to regular lapply to distribute jobs. Examples show using deathstar to estimate pi across nodes and calculate stock betas in a distributed manner. deathstar allows scaling R code to multiple cores locally or launching EC2 instances on AWS for more capacity.
Hierarchical free monads and software design in fp
I invented the approach I call "Hierarchical Free Monads". It helps to build applications in Haskell with achieving all the needed code quality requirements. I tested this approach in several real world projects and companies, and it works very well.
50k runs, millions of metrics, parameters or tags, some bursts at 20k QPS. That’s the volume of data managed by our MLflow tracking servers this year at Criteo. In this talk, you will learn how we set up a shared instance of MLflow at company scale. We will present our contributions to the SQLAlchemyStore to make it responsive at this scale. We will present you how we turned MLflow to a production-ready system. How we scaled horizontally a shared instance on a mesos cluster ? Our monitoring system based on prometheus. Integration to the company Single Sign-On (SSO) authentication. And how our data scientists register their runs from the largest hadoop cluster in Europe.
Node is used to build a reverse proxy to provide secure access to internal web resources and sites for mobile clients within a large enterprise. Performance testing shows the proxy can handle over 1000 requests per second with latency under 1 second. Code quality analysis tools like Plato and testing frameworks like Jest are useful for maintaining high quality code. Scalability is achieved through auto-scaling virtual machine instances with a load balancer and configuration management.
The document discusses optimizing the performance of Ruby on Rails applications. It covers optimizing Ruby code, Rails code, database queries, using alternative Ruby implementations like JRuby, and optimizing for production environments including shared filesystems, load balancing, and the frontend. Specific optimizations discussed include rewriting parts of the Date class in C, template inlining in Rails, pushing SQL conditions into subqueries, and using memcached instead of filesystem caching on a shared network.
The presentation has a quick preamble on SQL injection definition, sqlmap and its key features.
I will then illustrate into details common and uncommon problems and respective solutions with examples that a penetration tester faces when he wants to take advantage of any kind of web application SQL injection flaw on real world web applications, for instance SQL injection in ORDER BY and LIMIT clauses, single entry UNION query SQL injection, specific web application technologies IDS bypasses and more.
These slides have been presented at the 2nd Digital Security Forum in Lisbon on June 27, 2009.
Updated version of http://www.slideshare.net/inquis/sql-injection-not-only-and-11.
Expanding your impact with programmability in the data center
1) Digital readiness involves embracing automation, operational simplicity through programmability, network disaggregation, and DevOps to drive speed and agility.
2) Programmability allows for more flexibility, innovation, speed, repeatability and fewer mistakes by enabling automated workflows, off-box management via controllers and APIs, and SDKs for application development.
3) Adopting programmability means network operations will need to understand concepts like REST, XML, JSON, and work with developers to automate tasks and integrate the network.
SecZone 2011 - Cali, Colombia
(29th Nov. 2011)
SAP (in)security:
Scrubbing SAP clean with SOAP
------
Note
------
This is a slightly updated version of my Hashdays 2011 talk.
----------
Abstract:
----------
At the heart of any large enterprise, lies a platform misunderstood and feared by all but the bravest systems administrators. Home to a wealth of information, and key to infinite wisdom. This platform is SAP. For years this system has been amongst the many "red pen" items on penetration tests and audits alike... but no more! We will no longer accept the cries of "Business critical, out-of-scope". The time for SAP has come, the cross-hairs of attackers are firmly focused on the soft underbelly that is ERM, and it's our duty to follow suit. Join me as we take the first steps into exploring SAP, extracting information and popping shells. Leave your Nessus license at the door! It's time to scrub this SAP system clean with SOAP!
----------
The document discusses using R for analytics on Netezza's TwinFin appliance. TwinFin is a massively parallel processing database management system designed specifically for performance. It utilizes field programmable gate arrays and an "on-stream analytics" approach. The document outlines how R interfaces with TwinFin through functions like nzapply and nztapply that allow running R functions on TwinFin's distributed data in parallel. It provides examples of building decision trees and linear models on TwinFin tables using these functions.
This document discusses porting, scaling, and optimizing applications on Cray XT systems. It covers topics such as choosing compilers, profiling and debugging applications at scale, understanding CPU affinity, and improvements in the Cray Message Passing Toolkit (MPT). The document provides guidance on leveraging different compilers, collecting performance data using hardware counters and CrayPAT, understanding MPI process binding, and enhancements in MPT 4.0 related to MPI standards support and communication optimizations.
Better Network Management Through Network Programmability
As we enter the age of network programmability the data models, protocols, and tools provided by a programmable network can greatly improve and simplify network management tasks. Configuration and operational data can be read and set regardless of the underlying device. Errors are properly reported to ensure reliable delivery of data. Connections are secure and robust. Data is more intelligently extracted. This presentation will explore how tools like NETCONF, YANG, as well as Cisco's Embedded Event Manager, onePK APIs, and embedded Python scripting can radically improve network management applications by offering visibility and provisioning power throughout the network stack. For more information please visit our website here: http://www.cisco.com/web/CA/index.html
A talk on debugging of Python applications given at a local KharkivPy event.
A brief introduction into a set of tools that allow Python developers to debug common issues in their applications.
- The document discusses best practices for micro-benchmarking in Java, including using frameworks like JMH that account for JVM warmup and avoid benchmark overhead.
- It explains common pitfalls like dead code elimination and loop unrolling that can incorrectly optimize away the code being measured.
- An example benchmark compares the performance of ArrayList and LinkedList iteration in different Java versions.
String Comparison Surprises: Did Postgres lose my data?
Comparisons are fundamental to computing - and comparing strings is not nearly as straightforward as you might think. Come learn about the history, nuance and surprises of “putting words in order” that you never knew existed in computer science, and how that nuance impacts both general programming and SQL programming. Next, walk through a few actual scenarios and demonstrations using PostgreSQL as a user and administrator, which you can re-run yourself later for further study, including one way you could easily corrupt your self-managed PostgreSQL database if you aren't prepared. Finally we’ll dive into an explanation of the surprising behaviors we saw in PostgreSQL, and learn more about user and administrative features PostgreSQL provides related to localized string comparison.
The document discusses using the PERFORMANCE_SCHEMA feature in MySQL 5.6 to diagnose and improve the performance of a query that is not scaling well. It provides an example query against large tables that does not scale beyond 10 threads. Various tools for performance analysis are discussed, but the PERFORMANCE_SCHEMA is presented as a potentially better option for getting detailed insight into where time is being spent and how to optimize the server. The talk will cover both the capabilities and limitations of the PERFORMANCE_SCHEMA.
This document discusses various profiling tools that can be used to analyze MySQL performance, including Oprofile, perf, pt-pmp, and the MySQL Performance Schema. It provides examples of how these tools have been used to identify and resolve specific MySQL performance bugs. While the Performance Schema is useful, it does not always provide sufficient detail and other system-wide profilers like Oprofile and perf are still needed in some cases to pinpoint performance issues.
Entenda de onde vem toda a potência do Intel® Xeon Phi™
Palestra realizada por Luciano Palma no Intel Software Day 2013 (22/10/2013)
Conheça a arquitetura do Intel Xeon Phi, um coprocessador capaz de entregar mais de 2 TFlops de processamento para sua solução de HPC (High Performance Computing).
Beyond the Query – Bringing Complex Access Patterns to NoSQL with DataStax - ...
Learn how to model beyond traditional direct access in Apache Cassandra. Utilizing the DataStax platform to harness the power of Spark and Solr to perform search, analytics, and complex operations in place on your Cassandra data!
Keynote: Machine Learning for Design Automation at DAC 2018
Manish Pandey gave a keynote talk on transforming EDA with machine learning and discussed opportunities and challenges. He described how machine learning can be applied across different design abstraction levels from formal verification to silicon engineering. Pandey also discussed using machine learning techniques like reinforcement learning and word embeddings to optimize formal verification, simulation, and mask synthesis. Finally, he outlined challenges with data availability and model development for machine learning in EDA.
Advanced Defensive Coding Techniques (with Introduction to Design by Contract)
Structured exception handling and defensive programming are the two pillars of robust software.
Both pillars fail however when it comes to handling internal faults, those that normally originate in software defects rather than in any external factors.
In this webinar, Zoran Horvat demonstrates advanced defensive coding techniques that can bring the quality of your code to an entirely new level.
Watch the webinar and learn:
When throwing an exception is the right thing to do
Why exceptions and defensive coding cannot be applied to recover from defects
How to handle situations when internal software defect is causing the fault
How to treat fault detection as an orthogonal concern to normal operation
The webinar recording can be found here:
http://www.postsharp.net/blog/post/webinar-recording-defensive-programming
Applying Object Composition to Build Rich Domain Models
Modern business applications rely heavily on rich domain classes, which in turn rely heavily on polymorphic execution, code reuse and similar concepts.
How can we extend rich domain classes to support complex requirements?
In this presentation, Zoran Horvat will show why an object composition approach is favored over class inheritance when it comes to code reuse and polymorphism.
The presentation covers:
How class inheritance can lead to combinatorial explosion of classes
What the limitations of object composition are
What design patterns help consume composed objects
Techniques for creating rich features on composed objects
Watch the webinar recording here: http://www.postsharp.net/blog/post/webinar-recording-object-composition
The document provides an introduction to performance tuning. It discusses tracing SQL execution to analyze performance issues. Tracing can be done at different levels, and the tkprof utility helps analyze trace files by providing formatted output. Understanding execution plans is also an important part of performance tuning, as it shows the steps and cost of executing a SQL statement.
This document provides an in-depth overview of the LMS (Log Mining Server) process in Oracle databases. It discusses how LMS uses pollsys system calls and sockets to listen for incoming messages. It also examines the workload distribution across LMS processes and how LMS applies undo blocks to construct cache recovery (CR) buffers. Session-level statistics and tools like snapper.sql are demonstrated to analyze LMS workload and performance.
Verification of Concurrent and Distributed SystemsMykola Novik
Building correct concurrent and distributed systems is hard and very challenging task also high complexity of such software increases the probability of human error in design and architecture. On practice standard verification techniques in industry are necessary but not sufficient. In my talk we will discuss formal specification and verification language that helps engineers design, specify, reason about and verify complex, real-life algorithms and software systems.
Hidden Gems of Performance Tuning: Hierarchical Profiler and DML Trigger Opti...Michael Rosenblum
In any large ecosystem, there are always areas that stay in the twilight, outside of the public’s attention. This deep dive attempts to change the trend regarding two, at first glance, unrelated PL/SQL topics: hierarchical profiler (HProf) and database triggers. But if you look closer, there’s something in common: they’re significantly underused! HProf because nobody heard about it, database triggers because of decades-old stigma. Let’s put both of them back into our development toolset!
Part #1. One of the most critical FREE SQL and PL/SQL performance tuning tools is almost totally unknown! If you ask, how much time is spent on routine A? How often is function B called? Most developers would hand-code something instead of using the Oracle PL/SQL HProf. This isn’t because the provided functionality is disliked, but because developers aren’t aware of its existence! This presentation is an attempt to alter this trend and reintroduce HProf to a wider audience.
Part #2. There isn’t anything “evil” about database triggers; they just have to be used where they can actually solve problems. In this presentation, various kinds of triggers will be examined from a global system optimization view, including tradeoffs between multiple goals (e.g., depending upon the available hardware, developers can select either CPU-intensive or I/O-intensive solutions). This presentation will focus on the most common performance problems related to different kinds of DML triggers and the proper ways of resolving them.
Data Tracking: On the Hunt for Information about Your DatabaseMichael Rosenblum
Behind the scenes, Oracle databases hide a myriad of processes to ensure that your data can be safely stored and retrieved. These processes also leave “tracks” (or they COULD leave tracks if you set them up properly). These tracks, together with application-specific data, create a complete representation of the system’s day-to-day activity. Too often this representation is lost at the DBA/Developer borderline, mostly because one side is not aware of the needs of the other. This presentation strives to bridge this gap. It focuses on key sources of database information and techniques that are useful for both DBAs and developers:
- Data Dictionary
- Oracle Logging
- Oracle Tracing
- Advanced code instrumentation
This document discusses techniques for detecting and preventing SQL injection using the Percona Toolkit and Noinject!. It begins by introducing SQL injection and how attackers can modify SQL queries without changing server code. It then discusses using query fingerprints to detect new queries that may indicate injection attempts. The Percona Toolkit tools pt-query-digest and pt-fingerprint are used to generate and store fingerprints in a whitelist. Pt-query-digest can detect new fingerprints that have not been reviewed. The Noinject! proxy script uses fingerprints to inspect queries in real-time and block any that do not match whitelisted patterns. The document concludes by discussing limitations and ways to improve the fingerprinting approach.
The document discusses optimization of Real Application Clusters (RAC) in Oracle 12c. It provides background on the author and outlines common root causes of RAC performance issues such as CPU/memory starvation, network issues, and excessive dynamic remastering. The document then presents golden rules for RAC diagnostics including avoiding focusing only on top wait events, eliminating infrastructure issues, identifying problem instances, examining both send and receive side metrics, and using histograms. Specific techniques are described for analyzing wait events like gc buffer busy.
Flexviews is a materialized view solution for MySQL. This set of slides introduces Flexviews concepts, and gives some examples in how to use it and what to use it for.
This presentation is the attempt to switch sides and show code management from the developer's point of view. It stays outside of various VCS solutions and focuses on hands-on approaches: activity control via system triggers, conditional compilation, synonym manipulation, utilization of Edition-Based Redefinition (EBR).
Performance Schema for MySQL TroubleshootingSveta Smirnova
Percona Live (https://www.percona.com/live/data-performance-conference-2016/sessions/performance-schema-mysql-troubleshooting)
The performance schema in MySQL version 5.6, released in February, 2013, is a very powerful tool that can help DBAs discover why even the trickiest performance issues occur. Version 5.7 introduces even more instruments and tables. And while all these give you great power, you can get stuck choosing which instrument to use.
In this session, I will start with a description of a typical problem, then guide you how to use the performance schema to find out what causes the issue, the reason for unwanted behavior and how the received information can help you solve a particular problem.
Traditionally, performance schema sessions teach what is in contained in tables. I will, in contrast, start from a performance issue, then demonstrate which instruments and tables can help solve it. We will discuss how to setup the performance schema so that it has minimal impact on your server.
The document discusses tracing parallel execution in Oracle. It provides an example query using parallel execution hints and describes how to enable and generate trace files for the query coordinator process and parallel execution slave processes. It then summarizes the key events shown in the trace files, such as setting up parallel execution, full table scans by slave processes, and consolidating results.
QA your code: The new Unity Test Framework – Unite Copenhagen 2019Unity Technologies
Are you involved in testing or QA on projects in Unity? In these slides, you'll get an overview of the state of Unity for all things testing-related, and have the opportunity to share your stories of success, failure, pain, and glory. Learn from your fellow developers and give feedback on how Unity could help you hold your projects to a higher standard of quality. You will also get an introduction to the newest features in the Test Framework.
Speakers:
Christian Warnecke - Unity
Richard Fine - Unity
Watch the session on YouTube: https://youtu.be/wTiF2D0_vKA
The document discusses how the Oracle optimizer can sometimes choose suboptimal execution plans, leading to performance deterioration. It presents a scenario where the same query runs much slower when bind variables are used. The document then shows how SQL profiles can be used to enforce a better execution plan. It argues that manually creating profiles is not ideal for 24/7 environments. The document proposes using machine learning for outlier detection to identify performance issues and then automatically generate SQL profiles to address the issues. Code examples are provided for outlier detection and generating profiles through the Oracle API to allow automating the process.
A close encounter_with_real_world_and_odd_perf_issuesRiyaj Shamsudeen
This document discusses a performance issue where a database experienced high CPU usage in the kernel mode. Tracing tools identified that detaching from multiple shared memory segments during connection release was causing the high CPU. The database server had a NUMA architecture, causing the database instance to create multiple shared memory segments across NUMA nodes. Increasing the shared memory size limit did not resolve it, as the instance was optimizing for NUMA.
UKOUG version of a presentation trying to establish the sensible limits of parallelism on a couple of hardware configurations. Detailed white paper is at http://oracledoug.com/px_slaves.pdf
This document discusses SCAN, VIP, and HAIP in Oracle RAC environments. It provides details on:
- VIP - a virtual IP address that is not statically linked to a single node, allowing for faster failovers. Each node has a VIP.
- SCAN - a single virtual IP and listener that provides load balancing and high availability. SCAN acts as an abstraction layer so client connect strings do not need to change.
- HAIP - high availability IP addresses that allow clusterware and the database to use plumbed IP addresses for private interconnect traffic via solutions like bonding and trunking.
This document summarizes several myths about database redo, undo, commit, and rollback operations. It presents test cases and analysis to debunk the myths. The author is an experienced Oracle DBA who specializes in performance tuning and internals. Sample redo records are displayed and analyzed to explain how operations like rollback do generate redo. The document aims to clarify misunderstandings about the internal workings of Oracle's transaction and redo logging.
Database & Technology 1 _ Tom Kyte _ Efficient PL SQL - Why and How to Use.pdfInSync2011
Thomas Kyte discusses effective techniques for writing PL/SQL code. Some key points:
1) Use PL/SQL for data manipulation as it is tightly coupled with SQL and most efficient.
2) Write as little code as possible by leveraging SQL and thinking in sets rather than loops.
3) Use static SQL where possible for compile-time checking and dependency tracking. Dynamic SQL should only be used when static SQL is impractical.
4) Leverage packages to reduce dependencies, increase modularity, and support overloading and encapsulation.
5) Employ bulk processing techniques like bulk collects to minimize round trips to the database.
deathstar is a tool for seamlessly running R code across local and cloud computing nodes. It uses ZeroMQ to distribute data and code across nodes. The deathstar daemon runs on each node and listens for work. The R package exposes a zmq.cluster.lapply function similar to regular lapply to distribute jobs. Examples show using deathstar to estimate pi across nodes and calculate stock betas in a distributed manner. deathstar allows scaling R code to multiple cores locally or launching EC2 instances on AWS for more capacity.
Hierarchical free monads and software design in fpAlexander Granin
I invented the approach I call "Hierarchical Free Monads". It helps to build applications in Haskell with achieving all the needed code quality requirements. I tested this approach in several real world projects and companies, and it works very well.
50k runs, millions of metrics, parameters or tags, some bursts at 20k QPS. That’s the volume of data managed by our MLflow tracking servers this year at Criteo. In this talk, you will learn how we set up a shared instance of MLflow at company scale. We will present our contributions to the SQLAlchemyStore to make it responsive at this scale. We will present you how we turned MLflow to a production-ready system. How we scaled horizontally a shared instance on a mesos cluster ? Our monitoring system based on prometheus. Integration to the company Single Sign-On (SSO) authentication. And how our data scientists register their runs from the largest hadoop cluster in Europe.
Node is used to build a reverse proxy to provide secure access to internal web resources and sites for mobile clients within a large enterprise. Performance testing shows the proxy can handle over 1000 requests per second with latency under 1 second. Code quality analysis tools like Plato and testing frameworks like Jest are useful for maintaining high quality code. Scalability is achieved through auto-scaling virtual machine instances with a load balancer and configuration management.
Performance Optimization of Rails ApplicationsSerge Smetana
The document discusses optimizing the performance of Ruby on Rails applications. It covers optimizing Ruby code, Rails code, database queries, using alternative Ruby implementations like JRuby, and optimizing for production environments including shared filesystems, load balancing, and the frontend. Specific optimizations discussed include rewriting parts of the Date class in C, template inlining in Rails, pushing SQL conditions into subqueries, and using memcached instead of filesystem caching on a shared network.
The presentation has a quick preamble on SQL injection definition, sqlmap and its key features.
I will then illustrate into details common and uncommon problems and respective solutions with examples that a penetration tester faces when he wants to take advantage of any kind of web application SQL injection flaw on real world web applications, for instance SQL injection in ORDER BY and LIMIT clauses, single entry UNION query SQL injection, specific web application technologies IDS bypasses and more.
These slides have been presented at the 2nd Digital Security Forum in Lisbon on June 27, 2009.
Updated version of http://www.slideshare.net/inquis/sql-injection-not-only-and-11.
Expanding your impact with programmability in the data centerCisco Canada
1) Digital readiness involves embracing automation, operational simplicity through programmability, network disaggregation, and DevOps to drive speed and agility.
2) Programmability allows for more flexibility, innovation, speed, repeatability and fewer mistakes by enabling automated workflows, off-box management via controllers and APIs, and SDKs for application development.
3) Adopting programmability means network operations will need to understand concepts like REST, XML, JSON, and work with developers to automate tasks and integrate the network.
SecZone 2011 - Cali, Colombia
(29th Nov. 2011)
SAP (in)security:
Scrubbing SAP clean with SOAP
------
Note
------
This is a slightly updated version of my Hashdays 2011 talk.
----------
Abstract:
----------
At the heart of any large enterprise, lies a platform misunderstood and feared by all but the bravest systems administrators. Home to a wealth of information, and key to infinite wisdom. This platform is SAP. For years this system has been amongst the many "red pen" items on penetration tests and audits alike... but no more! We will no longer accept the cries of "Business critical, out-of-scope". The time for SAP has come, the cross-hairs of attackers are firmly focused on the soft underbelly that is ERM, and it's our duty to follow suit. Join me as we take the first steps into exploring SAP, extracting information and popping shells. Leave your Nessus license at the door! It's time to scrub this SAP system clean with SOAP!
----------
The document discusses using R for analytics on Netezza's TwinFin appliance. TwinFin is a massively parallel processing database management system designed specifically for performance. It utilizes field programmable gate arrays and an "on-stream analytics" approach. The document outlines how R interfaces with TwinFin through functions like nzapply and nztapply that allow running R functions on TwinFin's distributed data in parallel. It provides examples of building decision trees and linear models on TwinFin tables using these functions.
This document discusses porting, scaling, and optimizing applications on Cray XT systems. It covers topics such as choosing compilers, profiling and debugging applications at scale, understanding CPU affinity, and improvements in the Cray Message Passing Toolkit (MPT). The document provides guidance on leveraging different compilers, collecting performance data using hardware counters and CrayPAT, understanding MPI process binding, and enhancements in MPT 4.0 related to MPI standards support and communication optimizations.
Better Network Management Through Network ProgrammabilityCisco Canada
As we enter the age of network programmability the data models, protocols, and tools provided by a programmable network can greatly improve and simplify network management tasks. Configuration and operational data can be read and set regardless of the underlying device. Errors are properly reported to ensure reliable delivery of data. Connections are secure and robust. Data is more intelligently extracted. This presentation will explore how tools like NETCONF, YANG, as well as Cisco's Embedded Event Manager, onePK APIs, and embedded Python scripting can radically improve network management applications by offering visibility and provisioning power throughout the network stack. For more information please visit our website here: http://www.cisco.com/web/CA/index.html
A talk on debugging of Python applications given at a local KharkivPy event.
A brief introduction into a set of tools that allow Python developers to debug common issues in their applications.
- The document discusses best practices for micro-benchmarking in Java, including using frameworks like JMH that account for JVM warmup and avoid benchmark overhead.
- It explains common pitfalls like dead code elimination and loop unrolling that can incorrectly optimize away the code being measured.
- An example benchmark compares the performance of ArrayList and LinkedList iteration in different Java versions.
String Comparison Surprises: Did Postgres lose my data?Jeremy Schneider
Comparisons are fundamental to computing - and comparing strings is not nearly as straightforward as you might think. Come learn about the history, nuance and surprises of “putting words in order” that you never knew existed in computer science, and how that nuance impacts both general programming and SQL programming. Next, walk through a few actual scenarios and demonstrations using PostgreSQL as a user and administrator, which you can re-run yourself later for further study, including one way you could easily corrupt your self-managed PostgreSQL database if you aren't prepared. Finally we’ll dive into an explanation of the surprising behaviors we saw in PostgreSQL, and learn more about user and administrative features PostgreSQL provides related to localized string comparison.
The document discusses using the PERFORMANCE_SCHEMA feature in MySQL 5.6 to diagnose and improve the performance of a query that is not scaling well. It provides an example query against large tables that does not scale beyond 10 threads. Various tools for performance analysis are discussed, but the PERFORMANCE_SCHEMA is presented as a potentially better option for getting detailed insight into where time is being spent and how to optimize the server. The talk will cover both the capabilities and limitations of the PERFORMANCE_SCHEMA.
This document discusses various profiling tools that can be used to analyze MySQL performance, including Oprofile, perf, pt-pmp, and the MySQL Performance Schema. It provides examples of how these tools have been used to identify and resolve specific MySQL performance bugs. While the Performance Schema is useful, it does not always provide sufficient detail and other system-wide profilers like Oprofile and perf are still needed in some cases to pinpoint performance issues.
Palestra realizada por Luciano Palma no Intel Software Day 2013 (22/10/2013)
Conheça a arquitetura do Intel Xeon Phi, um coprocessador capaz de entregar mais de 2 TFlops de processamento para sua solução de HPC (High Performance Computing).
Beyond the Query – Bringing Complex Access Patterns to NoSQL with DataStax - ...StampedeCon
Learn how to model beyond traditional direct access in Apache Cassandra. Utilizing the DataStax platform to harness the power of Spark and Solr to perform search, analytics, and complex operations in place on your Cassandra data!
Keynote: Machine Learning for Design Automation at DAC 2018Manish Pandey
Manish Pandey gave a keynote talk on transforming EDA with machine learning and discussed opportunities and challenges. He described how machine learning can be applied across different design abstraction levels from formal verification to silicon engineering. Pandey also discussed using machine learning techniques like reinforcement learning and word embeddings to optimize formal verification, simulation, and mask synthesis. Finally, he outlined challenges with data availability and model development for machine learning in EDA.
Advanced Defensive Coding Techniques (with Introduction to Design by Contract)PostSharp Technologies
Structured exception handling and defensive programming are the two pillars of robust software.
Both pillars fail however when it comes to handling internal faults, those that normally originate in software defects rather than in any external factors.
In this webinar, Zoran Horvat demonstrates advanced defensive coding techniques that can bring the quality of your code to an entirely new level.
Watch the webinar and learn:
When throwing an exception is the right thing to do
Why exceptions and defensive coding cannot be applied to recover from defects
How to handle situations when internal software defect is causing the fault
How to treat fault detection as an orthogonal concern to normal operation
The webinar recording can be found here:
http://www.postsharp.net/blog/post/webinar-recording-defensive-programming
Modern business applications rely heavily on rich domain classes, which in turn rely heavily on polymorphic execution, code reuse and similar concepts.
How can we extend rich domain classes to support complex requirements?
In this presentation, Zoran Horvat will show why an object composition approach is favored over class inheritance when it comes to code reuse and polymorphism.
The presentation covers:
How class inheritance can lead to combinatorial explosion of classes
What the limitations of object composition are
What design patterns help consume composed objects
Techniques for creating rich features on composed objects
Watch the webinar recording here: http://www.postsharp.net/blog/post/webinar-recording-object-composition
How do you avoid the dreaded "this is not what we asked for" and ensure customer satisfaction when building a new system?
In this webinar, Dino Esposito demonstrates a top-down methodology, sometimes mistaken for plain common sense and often boldly ignored, called UX-Driven Design (UXDD).
UXDD means coming to a visual agreement with customers by using wireframing tools to iterate on sketches of the new system before building it. Then, rather than building the system from the data model, you proceed in a top-down fashion instead. The resulting system may be slow or even inefficient but it will never be the “wrong” system! In addition, UXDD leads to clarity on a few of today’s most popular patterns that are sometimes difficult to understand like CQRS and Event Sourcing.
The presentation covers:
An introduction to Wireframing tools
Proven ways to save on post-first deployment costs
Insights into better customer relationships
Better focus on development without 'analysis paralysis'
You can watch the webinar recording here:
http://www.postsharp.net/blog/post/webinar-recording-ux-driven-design
Localization is crucial for reaching out to a global audience, however, it’s often an afterthought for most developers and non-trivial to implement. Traditionally, game developers have outsourced this task due to its time consuming nature.
But it doesn’t have to be this way.
Yan Cui will show you a simple technique his team used at GameSys which allowed them to localize an entire story-driven, episodic MMORPG (with over 5000 items and 1500 quests) in under an hour of work and 50 lines of code, with the help of PostSharp.
When addressing website performance issues, developers typically jump to conclusions, focusing on the perceived causes rather than uncovering the real causes through research.
Mitchel Sellers will show you how to approach website performance issues with a level of consistency that ensures they're properly identified and resolved so you'll avoid jumping to conclusions in the future.
You can watch the webinar recording here:
https://www.postsharp.net/documentation/video?id=190066128
Developers spend up to 20% of their time writing repetitive code that machines could generate more reliably. This presentation explores the problem of duplicated source code that stems from manual implementation of patterns and reveals how to automate the boring side of programming and get a 19x ROI.
The presentation provides insight into:
- the problem of manual implementation of patterns, resulting in boilerplate code
- the cost of boilerplate for companies
- existing technologies for pattern automation
- the key reasons to consider pattern-aware compiler extensions
The white paper was written for CTOs, software architects and senior developers in software-driven organizations—specifically in financial, insurance, healthcare, energy and IT industries that typically write a lot of repetitive code.
Everybody knows the lock keyword, but how does it implemented? What are its performance characteristics. Gael Fraiteur scratches the surface of multithreaded programming in .NET and goes deep through the Windows Kernel down to CPU microarchitecture.
Gael Fraiteur argues that multithreaded programming needs to be addressed at the right level of abstraction, with design patterns like Actor, Immutable, Freezable, Thread Affine, Reader-Writer-Synchronized. Design patterns form a language and serve as a model against which code can be expressed. Compilers must support design patterns to allow code to be deterministically validated against the model, and of course to generate the low-level instructions that would be otherwise written manually.
This document discusses design pattern automation and the use of pattern-aware compilers. It provides a history of programming languages and patterns, defines design pattern automation as using tools to optimize productivity in implementing software based on patterns. It discusses how pattern-oriented software development involves implementing patterns at multiple levels of abstraction. It also addresses myths around why patterns cannot be automated and argues that emerging design and agile methodologies allow patterns to naturally emerge from iterative design processes. An example of applying automation to the Reader-Writer pattern through code generation and verification is also provided.
This document introduces aspect-oriented programming (AOP) and how it can be used to separate cross-cutting concerns from core business logic. It discusses how AOP frameworks like PostSharp allow aspects to modularize features like logging, transactions, and security. The document provides an example of how a rental booking method grows over time to include these concerns. It compares different AOP frameworks and concludes that PostSharp allows building aspects with strong robustness while minimizing code invasiveness.
COMPSAC 2024 D&I Panel: Charting a Course for Equity: Strategies for Overcomi...Hironori Washizaki
Hironori Washizaki, "Charting a Course for Equity: Strategies for Overcoming Challenges and Promoting Inclusion in the Metaverse", IEEE COMPSAC 2024 D&I Panel, 2024.
introduction of Ansys software and basic and advance knowledge of modelling s...sachin chaurasia
Ansys Mechanical enables you to solve complex structural engineering problems and make better, faster design decisions. With the finite element analysis (FEA) solvers available in the suite, you can customize and automate solutions for your structural mechanics problems and parameterize them to analyze multiple design scenarios. Ansys Mechanical is a dynamic tool that has a complete range of analysis tools.
Software development... for all? (keynote at ICSOFT'2024)miso_uam
Our world runs on software. It governs all major aspects of our life. It is an enabler for research and innovation, and is critical for business competitivity. Traditional software engineering techniques have achieved high effectiveness, but still may fall short on delivering software at the accelerated pace and with the increasing quality that future scenarios will require.
To attack this issue, some software paradigms raise the automation of software development via higher levels of abstraction through domain-specific languages (e.g., in model-driven engineering) and empowering non-professional developers with the possibility to build their own software (e.g., in low-code development approaches). In a software-demanding world, this is an attractive possibility, and perhaps -- paraphrasing Andy Warhol -- "in the future, everyone will be a developer for 15 minutes". However, to make this possible, methods are required to tweak languages to their context of use (crucial given the diversity of backgrounds and purposes), and the assistance to developers throughout the development process (especially critical for non-professionals).
In this keynote talk at ICSOFT'2024 I presented enabling techniques for this vision, supporting the creation of families of domain-specific languages, their adaptation to the usage context; and the augmentation of low-code environments with assistants and recommender systems to guide developers (professional or not) in the development process.
Sami provided a beginner-friendly introduction to Amazon Web Services (AWS), covering essential terms, products, and services for cloud deployment. Participants explored AWS' latest Gen AI offerings, making it accessible for those starting their cloud journey or integrating AI into coding practices.
CViewSurvey Digitech Pvt Ltd that works on a proven C.A.A.G. model.bhatinidhi2001
CViewSurvey is a SaaS-based Web & Mobile application that provides digital transformation to traditional paper surveys and feedback for customer & employee experience, field & market research that helps you evaluate your customer's as well as employee's loyalty.
With our unique C.A.A.G. Collect, Analysis, Act & Grow approach; business & industry’s can create customized surveys on web, publish on app to collect unlimited response & review AI backed real-time data analytics on mobile & tablets anytime, anywhere. Data collected when offline is securely stored in the device, which syncs to the cloud server when connected to any network.
Attendance Tracking From Paper To DigitalTask Tracker
If you are having trouble deciding which time tracker tool is best for you, try "Task Tracker" app. It has numerous features, including the ability to check daily attendance sheet, and other that make team management easier.
Responsibilities of Fleet Managers and How TrackoBit Can Assist.pdfTrackobit
What do fleet managers do? What are their duties, responsibilities, and challenges? And what makes a fleet manager effective and successful? This blog answers all these questions.
WhatsApp Tracker - Tracking WhatsApp to Boost Online Safety.pdfonemonitarsoftware
WhatsApp Tracker Software is an effective tool for remotely tracking the target’s WhatsApp activities. It allows users to monitor their loved one’s online behavior to ensure appropriate interactions for responsive device use.
Download this PPTX file and share this information to others.
9. Why?
“The most amazing achievement of
the computer software industry is its
continuing cancellation of the steady
and staggering gains made by the
computer hardware industry.”
- Henry Petroski
10. Why?
“We should forget about small efficiencies,
say about 97% of the time: premature
optimization is the root of all evil. Yet we
should not pass up our opportunities in
that critical 3%.“
- Donald Knuth
11. Why?
“We should forget about small efficiencies,
say about 97% of the time: premature
optimization is the root of all evil. Yet we
should not pass up our opportunities in
that critical 3%.“
- Donald Knuth
12. Never give up your
performance accidentally
Rico Mariani,
Performance Architect @
Microsoft
23. How?
“The simple act of putting a render time in the upper right hand corner of every
page we serve forced us to fix all our performance regressions and omissions.”
30. using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
static Uri @object = new Uri("http://google.com/search");
[Benchmark(Baseline = true)]
public string RegularPropertyCall()
{
return @object.Host;
}
[Benchmark]
public object Reflection()
{
Type @class = @object.GetType();
PropertyInfo property =
@class.GetProperty(propertyName, bindingFlags);
return property.GetValue(@object);
}
static void Main(string[] args)
{
var summary = BenchmarkRunner.Run<Program>();
}
31. Watch the webinar recording here:
http://www.postsharp.net/blog/pos
t/webinar-recording-performance-
is-a-feature
32. Compared to one second
• Millisecond – ms
–thousandth (0.001 or 1/1000)
• Microsecond - μs
–millionth (0.000001 or 1/1,000,000)
• Nanosecond - ns
–billionth (0.000000001 or 1/1,000,000,000)
33. BenchmarkDotNet
BenchmarkDotNet=v0.9.4.0
OS=Microsoft Windows NT 6.1.7601 Service Pack 1
Processor=Intel(R) Core(TM) i7-4800MQ CPU @ 2.70GHz, ProcessorCount=8
HostCLR=MS.NET 4.0.30319.42000, Arch=32-bit RELEASE
JitModules=clrjit-v4.6.100.0
Type=Program Mode=Throughput
Method | Median | StdDev | Scaled |
--------------------- |------------ |----------- |------- |
RegularPropertyCall |
Reflection |
35. [Params(1, 2, 3, 4, 5, 10, 100, 1000)]
public int Loops;
[Benchmark]
public string StringConcat()
{
string result = string.Empty;
for (int i = 0; i < Loops; ++i)
result = string.Concat(result, i.ToString());
return result;
}
[Benchmark]
public string StringBuilder()
{
StringBuilder sb = new StringBuilder(string.Empty);
for (int i = 0; i < Loops; ++i)
sb.Append(i.ToString());
return sb.ToString();
}
https://github.com/dotnet/roslyn/issues/5388
42. Stack Overflow Performance Lessons
Use static classes
Don’t be afraid to write your own tools
Dapper, Jil, MiniProfiler,
Intimately know your platform - CLR
44. Roslyn Performance Lessons 1
public class Logger
{
public static void WriteLine(string s) { /*...*/ }
}
public class Logger
{
public void Log(int id, int size)
{
var s = string.Format("{0}:{1}", id, size);
Logger.WriteLine(s);
}
}
Essential Truths Everyone Should Know about Performance in a Large Managed Codebase
http://channel9.msdn.com/Events/TechEd/NorthAmerica/2013/DEV-B333
45. Roslyn Performance Lessons 1
public class Logger
{
public static void WriteLine(string s) { /*...*/ }
}
public class BoxingExample
{
public void Log(int id, int size)
{
var s = string.Format("{0}:{1}",
id.ToString(), size.ToString());
Logger.WriteLine(s);
}
}
https://github.com/dotnet/roslyn/pull/415
AVOID BOXING
46. Roslyn Performance Lessons 2
class Symbol {
public string Name { get; private set; }
/*...*/
}
class Compiler {
private List<Symbol> symbols;
public Symbol FindMatchingSymbol(string name)
{
return symbols.FirstOrDefault(s => s.Name == name);
}
}
47. Roslyn Performance Lessons 2
class Symbol {
public string Name { get; private set; }
/*...*/
}
class Compiler {
private List<Symbol> symbols;
public Symbol FindMatchingSymbol(string name)
{
foreach (Symbol s in symbols)
{
if (s.Name == name)
return s;
}
return null;
}
}
DON’T USE LINQ
49. Roslyn Performance Lessons 3
public class Example
{
// Constructs a name like "Foo<T1, T2, T3>"
public string GenerateFullTypeName(string name, int arity)
{
StringBuilder sb = new StringBuilder();
sb.Append(name);
if (arity != 0)
{
sb.Append("<");
for (int i = 1; i < arity; i++)
{
sb.Append('T'); sb.Append(i.ToString());
}
sb.Append('T'); sb.Append(arity.ToString());
}
return sb.ToString();
}
}
50. Roslyn Performance Lessons 3
public class Example
{
// Constructs a name like "Foo<T1, T2, T3>"
public string GenerateFullTypeName(string name, int arity)
{
StringBuilder sb = new AcquireBuilder();
sb.Append(name);
if (arity != 0)
{
sb.Append("<");
for (int i = 1; i < arity; i++)
{
sb.Append('T'); sb.Append(i.ToString());
}
sb.Append('T'); sb.Append(arity.ToString());
}
return GetStringAndReleaseBuilder(sb);
}
}
OBJECT POOLING
53. Watch the webinar recording here:
http://www.postsharp.net/blog/pos
t/webinar-recording-performance-
is-a-feature
Editor's Notes
Who has: - any perf requirements - perf requirements with numbers! - any perf tests - perf test that are run continuously
I’m joined by Tony from PostSharp, ask questions on GoToWebinar question box (will be answered at the end)
Who has: - any perf requirements - perf requirements with numbers! - any perf tests - perf test that are run continuously
Front-end
- YSlow, Google PageSpeed, CDN & caching
- "High Performance Web Sites" by Steve Sounder
Database & caching
- Learn to use SQL Profiler
- Redis or similar
- MiniProfiler
.NET (server-side) <- This is what we are looking at
Mechanical Sympathy
- Anything by Martin Thompson
- Disruptor and Disruptor.NET
- CPU caches (L1, L2, etc)
- memory access patterns
Save money when running in the cloud (Zeeshan anecdote) - Scale-up rather than just scale-out- Save power on mobile devices (also bad perf more obvious on constrained device)- To users bad performance looks like you're website isn't working! - PerfBytes podcast, "News Of The Damned", a.k.a "which UK ticketing site has crashed this week"!- Bad performance might be losing you customers, before you even got them!!
- Even internal L.O.B apps - What could Dave in accounting do with an extra 50 minutes per week (10 min per/day) - Maybe the really slow accounting app is the reason for him quitting and going to work for your main competitor!!
Henry Petroski (February 6, 1942) is an American engineer specializing in failure analysis. A professor both of civil engineering and history at Duke University, he is also a prolific author.
To Engineer Is Human: The Role of Failure in Successful Design
To know the critical 3%, we have to measure,
Except Donal Knuth, who never write slow code and
if he did, he would know which bit was slow!
To know the critical 3%, we have to measure,
Except Donal Knuth, who never write slow code and
if he did, he would know which bit was slow!
Thanks him for making Visual Studio faster
He helped fix it after adding WPF made it SLOW!!!!
Should be roughly 10-15 mins in by now, if not hurry up!!!!
Normal distribution
Things like height, weight, DOESN’T apply to everything!!
Average is just less than 2, i.e. 1.995 or something like that
But > 99% of people in the UK have 2 legs (more than the average)
This is a histogram,
Real-world example
Web page response times
Why are there 2 groups of histograms bar?
- fast = cached data
- slow = hitting the database
Pause, for Tony to ask a question here (about next slide, ‘real-world’ example)
Unit tests are meant to be fast, and they only test 1 thing
In dev you don’t always have a full set of data
You don’t test for long periods of time
Smaller setup
Michelle Bustamante talk about logging, don’t just need to measure things,
Need to log the data AND be able to get at it!!
You’ll probably guess wrong!!
Consider adding performance unit tests,
Noda-Time does this, can graph performance over time, see if it’s regressed!!
MiniProfiler
Turn this on in Development and if possible in Production
Glimpse is an alternative
Runs on .NET,
Puts everything in 1 place, Web Server & Database
Summary metrics up front
Can drill-down into detailed metrics, including executed SQL, page load times, etc
Make sure you are really measuring what you think you are measuring!!
Question about profiler, fixing the most expensive things first, 80/20 rule?
Make sure you are really measuring what you think you are measuring!!
Question: how does it compare to other tools
Nbench
Xunit Performance
https://github.com/dotnet/roslyn/issues/5388
Implement string concatenation in loops via manipulating a StringBuilder instead of emitting String.Concat()
WON’T be implemented by the compiler
Question: when doing code review, should performance be taken into consideration
Both StackOverflow and Roslyn affected by this!!!!!
In the .NET Framework 4.5, there is background server garbage collection (before .NET 4.5 was Workstation only)
So until .NET 4.5, Server GC was STOP-THE-WORLD
Process Explorer
From Sysinternals
PerfView is a stand-alone utility, to help you debug CPU and memory problems
Light-weight and non-intrusive, can be used to on production apps with minimal impact
Uses ETW (Event Tracing for Windows), designed to be v. fast!!!!
They were able to graph these results & equate them to Garbage Collector pauses!!!
They had good logging and measurements in place,
They measured and found that all of these were on the HOT PATH
https://github.com/dotnet/roslyn/pull/415 Avoid unnecessary boxing with String.Concat
Able to implement this optimization for types which are immutable, pure, and not affected by other code. Notably:
- bool
- char (and this was one of the motivating types for this optimization)
- IntPtr
- UIntPtr
Due to side-effects of calling ToString() implementations that rely on the current culture (i.e. it culture can be changed mid-way through and you’ll see different behaviour)