SlideShare a Scribd company logo

Кирилл Безпалый, .NET Developer, Ciklum

•Download as PPTX, PDF•
•
A
Alina Vilk

The document discusses various topics related to Entity Framework (EF) and ORM usage in .NET development. It provides a brief history of ORM tools for .NET, examines common developer opinions on EF, discusses some performance and other technical issues with EF, and provides tips and examples for optimizing EF usage and queries. The document aims to help .NET developers understand both the benefits and challenges of using an ORM like EF in their projects.

Related slideshows

Useful and Practical Functionalities in Realm
Useful and Practical Functionalities in RealmUseful and Practical Functionalities in Realm
Useful and Practical Functionalities in Realm
 
Reactive Thinking in Java
Reactive Thinking in JavaReactive Thinking in Java
Reactive Thinking in Java
 
Lazy vs. Eager Loading Strategies in JPA 2.1
Lazy vs. Eager Loading Strategies in JPA 2.1Lazy vs. Eager Loading Strategies in JPA 2.1
Lazy vs. Eager Loading Strategies in JPA 2.1
 
1 of 39
ORM OR NOT ORM? THAT IS THE QUESTION BEZPALYI KYRYLO, .NET DEVELOPER @ CIKLUM
ABOUT ME • .Net developer @ Ciklum • Microsoft Student Partner • An avid programmer
BEFORE WE BEGIN ALL NON-TRIVIAL ABSTRACTIONS, TO SOME DEGREE, ARE LEAKY. Joel Spolsky, The Law of Leaky Abstractions YOU SHOULD ALWAYS UNDERSTAND AT LEAST ONE LAYER BELOW WHAT YOU ARE CODING. Lee Campbell, author of IntroToRX.com
SMALL HISTORY • ADO.NET • Data Sets • Strong-typed Data Sets • NHibernate • LINQ to SQL • Entity Framework • EF Code First • EF Core
WHAT IS .NET WAY TODAY? • .NET • ASP.NET (WebApi/MVC) • Sql Server • Entity Framework
WHAT’S .NET DEVELOPERS THINK ABOUT EF? • Too much configuration (Fluent API, attributes, navigation properties, keys, relations, etc.) • Leaking abstractions (LINQ) • Object model does not fit business model • Things become complicated very fast
Кирилл Безпалый, .NET Developer, Ciklum
PROBLEM OF ENTITY FRAMEWORK • Performance • Parameter Sniffing • Lazy Load • Data Modification • Some crazy stuff
Кирилл Безпалый, .NET Developer, Ciklum
HIDDEN POTENTIAN IQUERYABLE • Do you know difference between IEnumerable and IQueryable. Lot of tutorials don`t show full power of IQueryable showing only some simple examples. • In practice: IQueryable has generic logic of query using Expression Trees data structure and Provider that can transform this logic in query to some data source.
MORE ABOUT EXPRESSION TREE • Expression tree it’s a tree of operations and their operands which can be other operations • Generally Expression tree is a serialized code • Difference between IEnumerable and IQueryable: • IEnumerable represent sequence of values, values processed by delegates • IQueryable represent query collected with Expression Tree • Expressions has deep integration in VS, so in some cases programmers don’t understand that their lambda will be converted to some data structure instead of simple delegate. • Example: • Expression<Func<OrderLine, bool>> foo = (x) => x.Quantity > 10; • Looks like simple lambda, but left side says that compiler should use Expression Tree instead of delegate
COLD QUERY EXECUTION PROCESS Code User Writes Action Performance Impact using(var db = new WideWorldImporters()) { Context creation Low var q1 = from c in db. Orders where c.Id == 1000 select c; Query expression creation Low var c1 = q1.First(); LINQ query execution - Metadata loading: High but cached - View generation: Medium but cached - Parameter evaluation: Low - Query translation: Medium but cached - Materializer generation: Medium but cached - Database query execution: Potentially high (Better queries in some situations) + Connection.Open + Command.ExecuteReader + DataReader.Read Object materialization: Medium (Faster than EF5) - Identity lookup: Medium } Connection.Close Low
WARM QUERY EXECUTION PROCESS Code User Writes Action Performance Impact using(var db = new WideWorldImporters()) { Context creation Low var q1 = from c in db. Orders where c.Id == 1000 select c; Query expression creation Low var c1 = q1.First(); LINQ query execution - Metadata loading lookup: High but cached Low - View generation lookup: Medium but cached Low - Parameter evaluation: Low - Query translation lookup: Medium but cached Low - Materializer generation lookup: Medium but cached Low - Database query execution: Potentially high (Better queries in some situations) + Connection.Open + Command.ExecuteReader + DataReader.Read Object materialization: Medium (Faster than EF5) - Identity lookup: Medium } Connection.Close Low
SOME USEFUL OPTIMIZATIONS Find Find with AutoDetectChanges disabled Find with AutoDetectChanges disabled cached Find 85491 3525 39 1 10 100 1000 10000 100000 Using DbSet<T>.Find
HOW EF CACHE WORKS The cleanup algorithm is as follows: • Once the cache contains a set number of entries (800), we start a timer that periodically (once-per- minute) sweeps the cache. • During cache sweeps, entries are removed from the cache on a LFRU (Least frequently – recently used) basis. This algorithm takes both hit count and age into account when deciding which entries are ejected. • At the end of each cache sweep, the cache again contains 800 entries. All cache entries are treated equally when determining which entries to evict. This means the store command for a CompiledQuery has the same chance of eviction as the store command for an Entity SQL query. Note that the cache eviction timer is kicked in when there are 800 entities in the cache, but the cache is only swept 60 seconds after this timer is started. That means that for up to 60 seconds your cache may grow to be quite large.
CACHE TEST RESULTS Test EF6 no cache EF6 cached Enumerating all 18723 queries 124.3 125.3 Avoiding sweep (just the first 800 queries, regardless of complexity) 40.5 5.4 Just the AggregatingSubtotals queries (178 total - which avoids sweep) 38.1 4.6
CONTAINS<T> var ids = new int[10000]; ... using (var context = new WideWorldImporters("TestConString")) { var orders = context.Orders.Where(e => ids.Contains(e.CustomerID)).ToList(); ... }
PARAMETER SNIFFING using (var context = new WideWorldImporters()) { var myObject = new NonMappedType(); var query = from entity in context.Orders where entity.CustomerPurchaseOrderNumber .StartsWith(myObject.Mask) select entity; var results = query.ToList(); }
PARAMETER SNIFFING using (var context = new WideWorldImporters()) { var myObject = new NonMappedType(); var value = myObject.OrderNumberMask; var query = from entity in context.Orders where entity.CustomerPurchaseOrderNumber .StartsWith(value) select entity; var results = query.ToList(); }
CACHING AGAIN int[] ids = new int[10000]; using (var context = new WideWorldImporters()) { var firstQuery = from c in context.Customers where ids.Contains(c.CustomerId) select c; var secondQuery = from c in context.Customers where firstQuery.Any(otherEntity => otherEntity.CustomerId == c.CustomerId) select c; var results = secondQuery.ToList(); }
LAZY LOAD using (var context = new WideWorldImporters()) { var customers = context.Customers.Include(e=>e.Orders) .Where(c => c.BuyingGroup .BuyingGroupName == "Tailspin Toys"); var chosenCustomer = customers.First(); Console.WriteLine("Customer Id: {0} has {1} orders", chosenCustomer?.CustomerId, chosenCustomer?.Orders.Count); }
SELECT [Project1].[BuyingGroupID1] AS [BuyingGroupID], [Project1].[CustomerID] AS [CustomerID], [Project1].[CustomerName] AS [CustomerName], [Project1].[BillToCustomerID] AS [BillToCustomerID], [Project1].[CustomerCategoryID] AS [CustomerCategoryID], [Project1].[BuyingGroupID] AS [BuyingGroupID1], [Project1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Project1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Project1].[DeliveryMethodID] AS [DeliveryMethodID], [Project1].[DeliveryCityID] AS [DeliveryCityID], [Project1].[PostalCityID] AS [PostalCityID], [Project1].[CreditLimit] AS [CreditLimit], [Project1].[AccountOpenedDate] AS [AccountOpenedDate], [Project1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Project1].[IsStatementSent] AS [IsStatementSent], [Project1].[IsOnCreditHold] AS [IsOnCreditHold], [Project1].[PaymentDays] AS [PaymentDays], [Project1].[PhoneNumber] AS [PhoneNumber], [Project1].[FaxNumber] AS [FaxNumber], [Project1].[DeliveryRun] AS [DeliveryRun], [Project1].[RunPosition] AS [RunPosition], [Project1].[WebsiteURL] AS [WebsiteURL], [Project1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Project1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Project1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Project1].[PostalAddressLine1] AS [PostalAddressLine1], [Project1].[PostalAddressLine2] AS [PostalAddressLine2], [Project1].[PostalPostalCode] AS [PostalPostalCode], [Project1].[LastEditedBy] AS [LastEditedBy], [Project1].[ValidFrom] AS [ValidFrom], [Project1].[ValidTo] AS [ValidTo], [Project1].[C1] AS [C1], [Project1].[OrderID] AS [OrderID], [Project1].[CustomerID1] AS [CustomerID1], [Project1].[SalespersonPersonID] AS [SalespersonPersonID], [Project1].[PickedByPersonID] AS [PickedByPersonID], [Project1].[ContactPersonID] AS [ContactPersonID], [Project1].[BackorderOrderID] AS [BackorderOrderID], [Project1].[OrderDate] AS [OrderDate], [Project1].[ExpectedDeliveryDate] AS [ExpectedDeliveryDate], [Project1].[CustomerPurchaseOrderNumber] AS [CustomerPurchaseOrderNumber], [Project1].[IsUndersupplyBackordered] AS [IsUndersupplyBackordered], [Project1].[Comments] AS [Comments], [Project1].[DeliveryInstructions] AS [DeliveryInstructions], [Project1].[InternalComments] AS [InternalComments], [Project1].[PickingCompletedWhen] AS [PickingCompletedWhen], [Project1].[LastEditedBy1] AS [LastEditedBy1], [Project1].[LastEditedWhen] AS [LastEditedWhen] FROM ( SELECT [Limit1].[CustomerID] AS [CustomerID], [Limit1].[CustomerName] AS [CustomerName], [Limit1].[BillToCustomerID] AS [BillToCustomerID], [Limit1].[CustomerCategoryID] AS [CustomerCategoryID], [Limit1].[BuyingGroupID1] AS [BuyingGroupID], [Limit1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Limit1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Limit1].[DeliveryMethodID] AS [DeliveryMethodID], [Limit1].[DeliveryCityID] AS [DeliveryCityID], [Limit1].[PostalCityID] AS [PostalCityID], [Limit1].[CreditLimit] AS [CreditLimit], [Limit1].[AccountOpenedDate] AS [AccountOpenedDate], [Limit1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Limit1].[IsStatementSent] AS [IsStatementSent], [Limit1].[IsOnCreditHold] AS [IsOnCreditHold], [Limit1].[PaymentDays] AS [PaymentDays], [Limit1].[PhoneNumber] AS [PhoneNumber], [Limit1].[FaxNumber] AS [FaxNumber], [Limit1].[DeliveryRun] AS [DeliveryRun], [Limit1].[RunPosition] AS [RunPosition], [Limit1].[WebsiteURL] AS [WebsiteURL], [Limit1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Limit1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Limit1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Limit1].[PostalAddressLine1] AS [PostalAddressLine1], [Limit1].[PostalAddressLine2] AS [PostalAddressLine2], [Limit1].[PostalPostalCode] AS [PostalPostalCode], [Limit1].[LastEditedBy1] AS [LastEditedBy], [Limit1].[ValidFrom1] AS [ValidFrom], [Limit1].[ValidTo1] AS [ValidTo], [Limit1].[BuyingGroupID2] AS [BuyingGroupID1], [Extent3].[OrderID] AS [OrderID], [Extent3].[CustomerID] AS [CustomerID1], [Extent3].[SalespersonPersonID] AS [SalespersonPersonID], [Extent3].[PickedByPersonID] AS [PickedByPersonID], [Extent3].[ContactPersonID] AS [ContactPersonID], [Extent3].[BackorderOrderID] AS [BackorderOrderID], [Extent3].[OrderDate] AS [OrderDate], [Extent3].[ExpectedDeliveryDate] AS [ExpectedDeliveryDate], [Extent3].[CustomerPurchaseOrderNumber] AS [CustomerPurchaseOrderNumber], [Extent3].[IsUndersupplyBackordered] AS [IsUndersupplyBackordered], [Extent3].[Comments] AS [Comments], [Extent3].[DeliveryInstructions] AS [DeliveryInstructions], [Extent3].[InternalComments] AS [InternalComments], [Extent3].[PickingCompletedWhen] AS [PickingCompletedWhen], [Extent3].[LastEditedBy] AS [LastEditedBy1], [Extent3].[LastEditedWhen] AS [LastEditedWhen], CASE WHEN ([Extent3].[OrderID] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C1] FROM (SELECT TOP (1) [Extent1].[CustomerID] AS [CustomerID], [Extent1].[CustomerName] AS [CustomerName], [Extent1].[BillToCustomerID] AS [BillToCustomerID], [Extent1].[CustomerCategoryID] AS [CustomerCategoryID], [Extent1].[BuyingGroupID] AS [BuyingGroupID1], [Extent1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Extent1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Extent1].[DeliveryMethodID] AS [DeliveryMethodID], [Extent1].[DeliveryCityID] AS [DeliveryCityID], [Extent1].[PostalCityID] AS [PostalCityID], [Extent1].[CreditLimit] AS [CreditLimit], [Extent1].[AccountOpenedDate] AS [AccountOpenedDate], [Extent1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Extent1].[IsStatementSent] AS [IsStatementSent], [Extent1].[IsOnCreditHold] AS [IsOnCreditHold], [Extent1].[PaymentDays] AS [PaymentDays], [Extent1].[PhoneNumber] AS [PhoneNumber], [Extent1].[FaxNumber] AS [FaxNumber], [Extent1].[DeliveryRun] AS [DeliveryRun], [Extent1].[RunPosition] AS [RunPosition], [Extent1].[WebsiteURL] AS [WebsiteURL], [Extent1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Extent1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Extent1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Extent1].[PostalAddressLine1] AS [PostalAddressLine1], [Extent1].[PostalAddressLine2] AS [PostalAddressLine2], [Extent1].[PostalPostalCode] AS [PostalPostalCode], [Extent1].[LastEditedBy] AS [LastEditedBy1], [Extent1].[ValidFrom] AS [ValidFrom1], [Extent1].[ValidTo] AS [ValidTo1], [Extent2].[BuyingGroupID] AS [BuyingGroupID2] FROM [Sales].[Customers] AS [Extent1] INNER JOIN [Sales].[BuyingGroups] AS [Extent2] ON [Extent1].[BuyingGroupID] = [Extent2].[BuyingGroupID] WHERE N'Tailspin Toys' = [Extent2].[BuyingGroupName] ) AS [Limit1] LEFT OUTER JOIN [Sales].[Orders] AS [Extent3] ON [Limit1].[CustomerID] = [Extent3].[CustomerID] ) AS [Project1] ORDER BY [Project1].[BuyingGroupID1] ASC, [Project1].[CustomerID] ASC, [Project1].[C1] ASC
LAZY LOAD using (var context = new WideWorldImporters()) { var customers = context.Customers .Where(c => c.BuyingGroup .BuyingGroupName == "Tailspin Toys"); var chosenCustomer = customers.First(); Console.WriteLine("Customer Id: {0} has {1} orders", chosenCustomer?.CustomerId, chosenCustomer?.Orders.Count); }
SELECT TOP (1) [Extent1].[CustomerID] AS [CustomerID], [Extent1].[CustomerName] AS [CustomerName], [Extent1].[BillToCustomerID] AS [BillToCustomerID], [Extent1].[CustomerCategoryID] AS [CustomerCategoryID], [Extent1].[BuyingGroupID] AS [BuyingGroupID], [Extent1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Extent1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Extent1].[DeliveryMethodID] AS [DeliveryMethodID], [Extent1].[DeliveryCityID] AS [DeliveryCityID], [Extent1].[PostalCityID] AS [PostalCityID], [Extent1].[CreditLimit] AS [CreditLimit], [Extent1].[AccountOpenedDate] AS [AccountOpenedDate], [Extent1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Extent1].[IsStatementSent] AS [IsStatementSent], [Extent1].[IsOnCreditHold] AS [IsOnCreditHold], [Extent1].[PaymentDays] AS [PaymentDays], [Extent1].[PhoneNumber] AS [PhoneNumber], [Extent1].[FaxNumber] AS [FaxNumber], [Extent1].[DeliveryRun] AS [DeliveryRun], [Extent1].[RunPosition] AS [RunPosition], [Extent1].[WebsiteURL] AS [WebsiteURL], [Extent1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Extent1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Extent1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Extent1].[PostalAddressLine1] AS [PostalAddressLine1], [Extent1].[PostalAddressLine2] AS [PostalAddressLine2], [Extent1].[PostalPostalCode] AS [PostalPostalCode], [Extent1].[LastEditedBy] AS [LastEditedBy], [Extent1].[ValidFrom] AS [ValidFrom], [Extent1].[ValidTo] AS [ValidTo] FROM [Sales].[Customers] AS [Extent1] INNER JOIN [Sales].[BuyingGroups] AS [Extent2] ON [Extent1].[BuyingGroupID] = [Extent2].[BuyingGroupID] WHERE N'Tailspin Toys' = [Extent2].[BuyingGroupName] exec sp_executesql N'SELECT [Extent1].[OrderID] AS [OrderID], [Extent1].[CustomerID] AS [CustomerID], [Extent1].[SalespersonPersonID] AS [SalespersonPersonID], [Extent1].[PickedByPersonID] AS [PickedByPersonID], [Extent1].[ContactPersonID] AS [ContactPersonID], [Extent1].[BackorderOrderID] AS [BackorderOrderID], [Extent1].[OrderDate] AS [OrderDate], [Extent1].[ExpectedDeliveryDate] AS [ExpectedDeliveryDate], [Extent1].[CustomerPurchaseOrderNumber] AS [CustomerPurchaseOrderNumber], [Extent1].[IsUndersupplyBackordered] AS [IsUndersupplyBackordered], [Extent1].[Comments] AS [Comments], [Extent1].[DeliveryInstructions] AS [DeliveryInstructions], [Extent1].[InternalComments] AS [InternalComments], [Extent1].[PickingCompletedWhen] AS [PickingCompletedWhen], [Extent1].[LastEditedBy] AS [LastEditedBy], [Extent1].[LastEditedWhen] AS [LastEditedWhen] FROM [Sales].[Orders] AS [Extent1] WHERE [Extent1].[CustomerID] = @EntityKeyValue1',N'@EntityKeyValue1 int',@EntityKeyValue1=1
DATA MODIFICATION context.Customers .Where(t => t.Name == “Petya”) .Update(t => new Customer { Name = “Petya” }); PM> Install-Package EntityFramework.Extended context.BulkInsert(customers); PM> Install-Package Z.EntityFramework.Extensions
OOP public class WideWorldImporters : DbContext { public WideWorldImporters(string conString) : base(conString) { } // Some DbSets }
OOP public class WideWorldImporters : DbContext { public WideWorldImporters(string conString) : base(conString) { } public WideWorldImporters() : this("WideWorldImporters") { } // Some DbSets } using (var context = new WideWorldImport ers("TestConString")) { context.Orders.ToList(); }
IF ORM IS SO BAD, WHAT’S SOLUTION? Micro-ORM • Handcraft SQL • Handle mappings • Less code (less functionality)
DAPPER • Created for Stack Overflow • Very. VERY fast • Works with POCO and dynamics using (var connection = new SqlConnection(conString)) { var users = connection.Query<User>("select * from Users") .ToList(); } using (var connection = new SqlConnection(conString)) { IEnumerable<dynamic> users = connection.Query ("select * from Users") .ToList(); }
DAPPER. MAPPING var sql = @"select * from Posts p left join Users u on u.Id = p.OwnerId order by p.Id"; var data = connection.Query<Post, User, Post>( sql, (post, user) => { post.Owner = user; return post;}); var posts = data.ToList();
DAPPER. MULTI RESULTS var sql = @" select * from Customers where CustomerId = @id select * from Orders where CustomerId = @id select * from Returns where CustomerId = @id"; using (var multi = connection.QueryMultiple(sql, new {id=selectedId})) { var customer = multi.Read<Customer>().Single(); var orders = multi.Read<Order>().ToList(); var returns = multi.Read<Return>().ToList(); }
DAPPER. PROCEDURES var user = cnn.Query<User>( "sp_GetUser", new {Id = 1}, commandType: CommandType.StoredProcedure) .SingleOrDefault();
DAPPER. OUTPUT PARAMETERS var p = new DynamicParameters(); p.Add("@a", 11); p.Add("@b", dbType: DbType.Int32, direction: ParameterDirection.Output); p.Add("@c", dbType: DbType.Int32, direction: ParameterDirection.ReturnValue); con.Execute( "spMagicProc", p, commandType: CommandType.StoredProcedure); int b = p.Get<int>("@b"); int c = p.Get<int>("@c");
PETAPOCO Fast like Dapper Works with strictly undecorated POCOs, or attributed almost-POCOs var user1 = db.Single<User>(1); var user2 = db.Single<User>("WHERE Name = @0", “Vasya");
PETAPOCO var u = new User(); u.Name = “Vasya”; db.Insert(u); // Update it u.Name = “Kolya"; db.Update(u); // Delete it db.Delete(u); db.Delete<User>( "WHERE Name LIKE @0", “V%"); db.Update<User>( "SET Name=@0 WHERE Id>@1", “XXX", 10);
SIMPLE DATA Fun project of @markrendle Seems to be dead But still is fun return Database.Open() .Users.FindAllByName(name) .FirstOrDefault();
PERFORMANCE SELECT MAPPING OVER 500 ITERATIONS - POCO SERIALIZATION Method Duration Hand coded (using a SqlDataReader) 47ms Dapper ExecuteMapperQuery 49ms PetaPoco 52ms NHibernate SQL 104ms Linq 2 SQL ExecuteQuery 181ms Entity framework ExecuteStoreQuery 631ms SELECT MAPPING OVER 500 ITERATIONS - DYNAMIC SERIALIZATION Method Duration Dapper ExecuteMapperQuery (dynamic) 48ms Simple.Data 95ms
CONCLUSIONS • ORM good or bad? Depends. • Micro-ORM is good alternative in cases when ORM is bad. • Knowing internal implementations or behavior can increase performance of using of the frameworks. • You should always understand at least one layer below what you are coding.
Кирилл Безпа��ый, .NET Developer, Ciklum

More Related Content

Кирилл Безпалый, .NET Developer, Ciklum

  • 1. ORM OR NOT ORM? THAT IS THE QUESTION BEZPALYI KYRYLO, .NET DEVELOPER @ CIKLUM
  • 2. ABOUT ME • .Net developer @ Ciklum • Microsoft Student Partner • An avid programmer
  • 3. BEFORE WE BEGIN ALL NON-TRIVIAL ABSTRACTIONS, TO SOME DEGREE, ARE LEAKY. Joel Spolsky, The Law of Leaky Abstractions YOU SHOULD ALWAYS UNDERSTAND AT LEAST ONE LAYER BELOW WHAT YOU ARE CODING. Lee Campbell, author of IntroToRX.com
  • 4. SMALL HISTORY • ADO.NET • Data Sets • Strong-typed Data Sets • NHibernate • LINQ to SQL • Entity Framework • EF Code First • EF Core
  • 5. WHAT IS .NET WAY TODAY? • .NET • ASP.NET (WebApi/MVC) • Sql Server • Entity Framework
  • 6. WHAT’S .NET DEVELOPERS THINK ABOUT EF? • Too much configuration (Fluent API, attributes, navigation properties, keys, relations, etc.) • Leaking abstractions (LINQ) • Object model does not fit business model • Things become complicated very fast
  • 8. PROBLEM OF ENTITY FRAMEWORK • Performance • Parameter Sniffing • Lazy Load • Data Modification • Some crazy stuff
  • 10. HIDDEN POTENTIAN IQUERYABLE • Do you know difference between IEnumerable and IQueryable. Lot of tutorials don`t show full power of IQueryable showing only some simple examples. • In practice: IQueryable has generic logic of query using Expression Trees data structure and Provider that can transform this logic in query to some data source.
  • 11. MORE ABOUT EXPRESSION TREE • Expression tree it’s a tree of operations and their operands which can be other operations • Generally Expression tree is a serialized code • Difference between IEnumerable and IQueryable: • IEnumerable represent sequence of values, values processed by delegates • IQueryable represent query collected with Expression Tree • Expressions has deep integration in VS, so in some cases programmers don’t understand that their lambda will be converted to some data structure instead of simple delegate. • Example: • Expression<Func<OrderLine, bool>> foo = (x) => x.Quantity > 10; • Looks like simple lambda, but left side says that compiler should use Expression Tree instead of delegate
  • 12. COLD QUERY EXECUTION PROCESS Code User Writes Action Performance Impact using(var db = new WideWorldImporters()) { Context creation Low var q1 = from c in db. Orders where c.Id == 1000 select c; Query expression creation Low var c1 = q1.First(); LINQ query execution - Metadata loading: High but cached - View generation: Medium but cached - Parameter evaluation: Low - Query translation: Medium but cached - Materializer generation: Medium but cached - Database query execution: Potentially high (Better queries in some situations) + Connection.Open + Command.ExecuteReader + DataReader.Read Object materialization: Medium (Faster than EF5) - Identity lookup: Medium } Connection.Close Low
  • 13. WARM QUERY EXECUTION PROCESS Code User Writes Action Performance Impact using(var db = new WideWorldImporters()) { Context creation Low var q1 = from c in db. Orders where c.Id == 1000 select c; Query expression creation Low var c1 = q1.First(); LINQ query execution - Metadata loading lookup: High but cached Low - View generation lookup: Medium but cached Low - Parameter evaluation: Low - Query translation lookup: Medium but cached Low - Materializer generation lookup: Medium but cached Low - Database query execution: Potentially high (Better queries in some situations) + Connection.Open + Command.ExecuteReader + DataReader.Read Object materialization: Medium (Faster than EF5) - Identity lookup: Medium } Connection.Close Low
  • 14. SOME USEFUL OPTIMIZATIONS Find Find with AutoDetectChanges disabled Find with AutoDetectChanges disabled cached Find 85491 3525 39 1 10 100 1000 10000 100000 Using DbSet<T>.Find
  • 15. HOW EF CACHE WORKS The cleanup algorithm is as follows: • Once the cache contains a set number of entries (800), we start a timer that periodically (once-per- minute) sweeps the cache. • During cache sweeps, entries are removed from the cache on a LFRU (Least frequently – recently used) basis. This algorithm takes both hit count and age into account when deciding which entries are ejected. • At the end of each cache sweep, the cache again contains 800 entries. All cache entries are treated equally when determining which entries to evict. This means the store command for a CompiledQuery has the same chance of eviction as the store command for an Entity SQL query. Note that the cache eviction timer is kicked in when there are 800 entities in the cache, but the cache is only swept 60 seconds after this timer is started. That means that for up to 60 seconds your cache may grow to be quite large.
  • 16. CACHE TEST RESULTS Test EF6 no cache EF6 cached Enumerating all 18723 queries 124.3 125.3 Avoiding sweep (just the first 800 queries, regardless of complexity) 40.5 5.4 Just the AggregatingSubtotals queries (178 total - which avoids sweep) 38.1 4.6
  • 17. CONTAINS<T> var ids = new int[10000]; ... using (var context = new WideWorldImporters("TestConString")) { var orders = context.Orders.Where(e => ids.Contains(e.CustomerID)).ToList(); ... }
  • 18. PARAMETER SNIFFING using (var context = new WideWorldImporters()) { var myObject = new NonMappedType(); var query = from entity in context.Orders where entity.CustomerPurchaseOrderNumber .StartsWith(myObject.Mask) select entity; var results = query.ToList(); }
  • 19. PARAMETER SNIFFING using (var context = new WideWorldImporters()) { var myObject = new NonMappedType(); var value = myObject.OrderNumberMask; var query = from entity in context.Orders where entity.CustomerPurchaseOrderNumber .StartsWith(value) select entity; var results = query.ToList(); }
  • 20. CACHING AGAIN int[] ids = new int[10000]; using (var context = new WideWorldImporters()) { var firstQuery = from c in context.Customers where ids.Contains(c.CustomerId) select c; var secondQuery = from c in context.Customers where firstQuery.Any(otherEntity => otherEntity.CustomerId == c.CustomerId) select c; var results = secondQuery.ToList(); }
  • 21. LAZY LOAD using (var context = new WideWorldImporters()) { var customers = context.Customers.Include(e=>e.Orders) .Where(c => c.BuyingGroup .BuyingGroupName == "Tailspin Toys"); var chosenCustomer = customers.First(); Console.WriteLine("Customer Id: {0} has {1} orders", chosenCustomer?.CustomerId, chosenCustomer?.Orders.Count); }
  • 22. SELECT [Project1].[BuyingGroupID1] AS [BuyingGroupID], [Project1].[CustomerID] AS [CustomerID], [Project1].[CustomerName] AS [CustomerName], [Project1].[BillToCustomerID] AS [BillToCustomerID], [Project1].[CustomerCategoryID] AS [CustomerCategoryID], [Project1].[BuyingGroupID] AS [BuyingGroupID1], [Project1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Project1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Project1].[DeliveryMethodID] AS [DeliveryMethodID], [Project1].[DeliveryCityID] AS [DeliveryCityID], [Project1].[PostalCityID] AS [PostalCityID], [Project1].[CreditLimit] AS [CreditLimit], [Project1].[AccountOpenedDate] AS [AccountOpenedDate], [Project1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Project1].[IsStatementSent] AS [IsStatementSent], [Project1].[IsOnCreditHold] AS [IsOnCreditHold], [Project1].[PaymentDays] AS [PaymentDays], [Project1].[PhoneNumber] AS [PhoneNumber], [Project1].[FaxNumber] AS [FaxNumber], [Project1].[DeliveryRun] AS [DeliveryRun], [Project1].[RunPosition] AS [RunPosition], [Project1].[WebsiteURL] AS [WebsiteURL], [Project1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Project1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Project1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Project1].[PostalAddressLine1] AS [PostalAddressLine1], [Project1].[PostalAddressLine2] AS [PostalAddressLine2], [Project1].[PostalPostalCode] AS [PostalPostalCode], [Project1].[LastEditedBy] AS [LastEditedBy], [Project1].[ValidFrom] AS [ValidFrom], [Project1].[ValidTo] AS [ValidTo], [Project1].[C1] AS [C1], [Project1].[OrderID] AS [OrderID], [Project1].[CustomerID1] AS [CustomerID1], [Project1].[SalespersonPersonID] AS [SalespersonPersonID], [Project1].[PickedByPersonID] AS [PickedByPersonID], [Project1].[ContactPersonID] AS [ContactPersonID], [Project1].[BackorderOrderID] AS [BackorderOrderID], [Project1].[OrderDate] AS [OrderDate], [Project1].[ExpectedDeliveryDate] AS [ExpectedDeliveryDate], [Project1].[CustomerPurchaseOrderNumber] AS [CustomerPurchaseOrderNumber], [Project1].[IsUndersupplyBackordered] AS [IsUndersupplyBackordered], [Project1].[Comments] AS [Comments], [Project1].[DeliveryInstructions] AS [DeliveryInstructions], [Project1].[InternalComments] AS [InternalComments], [Project1].[PickingCompletedWhen] AS [PickingCompletedWhen], [Project1].[LastEditedBy1] AS [LastEditedBy1], [Project1].[LastEditedWhen] AS [LastEditedWhen] FROM ( SELECT [Limit1].[CustomerID] AS [CustomerID], [Limit1].[CustomerName] AS [CustomerName], [Limit1].[BillToCustomerID] AS [BillToCustomerID], [Limit1].[CustomerCategoryID] AS [CustomerCategoryID], [Limit1].[BuyingGroupID1] AS [BuyingGroupID], [Limit1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Limit1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Limit1].[DeliveryMethodID] AS [DeliveryMethodID], [Limit1].[DeliveryCityID] AS [DeliveryCityID], [Limit1].[PostalCityID] AS [PostalCityID], [Limit1].[CreditLimit] AS [CreditLimit], [Limit1].[AccountOpenedDate] AS [AccountOpenedDate], [Limit1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Limit1].[IsStatementSent] AS [IsStatementSent], [Limit1].[IsOnCreditHold] AS [IsOnCreditHold], [Limit1].[PaymentDays] AS [PaymentDays], [Limit1].[PhoneNumber] AS [PhoneNumber], [Limit1].[FaxNumber] AS [FaxNumber], [Limit1].[DeliveryRun] AS [DeliveryRun], [Limit1].[RunPosition] AS [RunPosition], [Limit1].[WebsiteURL] AS [WebsiteURL], [Limit1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Limit1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Limit1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Limit1].[PostalAddressLine1] AS [PostalAddressLine1], [Limit1].[PostalAddressLine2] AS [PostalAddressLine2], [Limit1].[PostalPostalCode] AS [PostalPostalCode], [Limit1].[LastEditedBy1] AS [LastEditedBy], [Limit1].[ValidFrom1] AS [ValidFrom], [Limit1].[ValidTo1] AS [ValidTo], [Limit1].[BuyingGroupID2] AS [BuyingGroupID1], [Extent3].[OrderID] AS [OrderID], [Extent3].[CustomerID] AS [CustomerID1], [Extent3].[SalespersonPersonID] AS [SalespersonPersonID], [Extent3].[PickedByPersonID] AS [PickedByPersonID], [Extent3].[ContactPersonID] AS [ContactPersonID], [Extent3].[BackorderOrderID] AS [BackorderOrderID], [Extent3].[OrderDate] AS [OrderDate], [Extent3].[ExpectedDeliveryDate] AS [ExpectedDeliveryDate], [Extent3].[CustomerPurchaseOrderNumber] AS [CustomerPurchaseOrderNumber], [Extent3].[IsUndersupplyBackordered] AS [IsUndersupplyBackordered], [Extent3].[Comments] AS [Comments], [Extent3].[DeliveryInstructions] AS [DeliveryInstructions], [Extent3].[InternalComments] AS [InternalComments], [Extent3].[PickingCompletedWhen] AS [PickingCompletedWhen], [Extent3].[LastEditedBy] AS [LastEditedBy1], [Extent3].[LastEditedWhen] AS [LastEditedWhen], CASE WHEN ([Extent3].[OrderID] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C1] FROM (SELECT TOP (1) [Extent1].[CustomerID] AS [CustomerID], [Extent1].[CustomerName] AS [CustomerName], [Extent1].[BillToCustomerID] AS [BillToCustomerID], [Extent1].[CustomerCategoryID] AS [CustomerCategoryID], [Extent1].[BuyingGroupID] AS [BuyingGroupID1], [Extent1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Extent1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Extent1].[DeliveryMethodID] AS [DeliveryMethodID], [Extent1].[DeliveryCityID] AS [DeliveryCityID], [Extent1].[PostalCityID] AS [PostalCityID], [Extent1].[CreditLimit] AS [CreditLimit], [Extent1].[AccountOpenedDate] AS [AccountOpenedDate], [Extent1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Extent1].[IsStatementSent] AS [IsStatementSent], [Extent1].[IsOnCreditHold] AS [IsOnCreditHold], [Extent1].[PaymentDays] AS [PaymentDays], [Extent1].[PhoneNumber] AS [PhoneNumber], [Extent1].[FaxNumber] AS [FaxNumber], [Extent1].[DeliveryRun] AS [DeliveryRun], [Extent1].[RunPosition] AS [RunPosition], [Extent1].[WebsiteURL] AS [WebsiteURL], [Extent1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Extent1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Extent1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Extent1].[PostalAddressLine1] AS [PostalAddressLine1], [Extent1].[PostalAddressLine2] AS [PostalAddressLine2], [Extent1].[PostalPostalCode] AS [PostalPostalCode], [Extent1].[LastEditedBy] AS [LastEditedBy1], [Extent1].[ValidFrom] AS [ValidFrom1], [Extent1].[ValidTo] AS [ValidTo1], [Extent2].[BuyingGroupID] AS [BuyingGroupID2] FROM [Sales].[Customers] AS [Extent1] INNER JOIN [Sales].[BuyingGroups] AS [Extent2] ON [Extent1].[BuyingGroupID] = [Extent2].[BuyingGroupID] WHERE N'Tailspin Toys' = [Extent2].[BuyingGroupName] ) AS [Limit1] LEFT OUTER JOIN [Sales].[Orders] AS [Extent3] ON [Limit1].[CustomerID] = [Extent3].[CustomerID] ) AS [Project1] ORDER BY [Project1].[BuyingGroupID1] ASC, [Project1].[CustomerID] ASC, [Project1].[C1] ASC
  • 23. LAZY LOAD using (var context = new WideWorldImporters()) { var customers = context.Customers .Where(c => c.BuyingGroup .BuyingGroupName == "Tailspin Toys"); var chosenCustomer = customers.First(); Console.WriteLine("Customer Id: {0} has {1} orders", chosenCustomer?.CustomerId, chosenCustomer?.Orders.Count); }
  • 24. SELECT TOP (1) [Extent1].[CustomerID] AS [CustomerID], [Extent1].[CustomerName] AS [CustomerName], [Extent1].[BillToCustomerID] AS [BillToCustomerID], [Extent1].[CustomerCategoryID] AS [CustomerCategoryID], [Extent1].[BuyingGroupID] AS [BuyingGroupID], [Extent1].[PrimaryContactPersonID] AS [PrimaryContactPersonID], [Extent1].[AlternateContactPersonID] AS [AlternateContactPersonID], [Extent1].[DeliveryMethodID] AS [DeliveryMethodID], [Extent1].[DeliveryCityID] AS [DeliveryCityID], [Extent1].[PostalCityID] AS [PostalCityID], [Extent1].[CreditLimit] AS [CreditLimit], [Extent1].[AccountOpenedDate] AS [AccountOpenedDate], [Extent1].[StandardDiscountPercentage] AS [StandardDiscountPercentage], [Extent1].[IsStatementSent] AS [IsStatementSent], [Extent1].[IsOnCreditHold] AS [IsOnCreditHold], [Extent1].[PaymentDays] AS [PaymentDays], [Extent1].[PhoneNumber] AS [PhoneNumber], [Extent1].[FaxNumber] AS [FaxNumber], [Extent1].[DeliveryRun] AS [DeliveryRun], [Extent1].[RunPosition] AS [RunPosition], [Extent1].[WebsiteURL] AS [WebsiteURL], [Extent1].[DeliveryAddressLine1] AS [DeliveryAddressLine1], [Extent1].[DeliveryAddressLine2] AS [DeliveryAddressLine2], [Extent1].[DeliveryPostalCode] AS [DeliveryPostalCode], [Extent1].[PostalAddressLine1] AS [PostalAddressLine1], [Extent1].[PostalAddressLine2] AS [PostalAddressLine2], [Extent1].[PostalPostalCode] AS [PostalPostalCode], [Extent1].[LastEditedBy] AS [LastEditedBy], [Extent1].[ValidFrom] AS [ValidFrom], [Extent1].[ValidTo] AS [ValidTo] FROM [Sales].[Customers] AS [Extent1] INNER JOIN [Sales].[BuyingGroups] AS [Extent2] ON [Extent1].[BuyingGroupID] = [Extent2].[BuyingGroupID] WHERE N'Tailspin Toys' = [Extent2].[BuyingGroupName] exec sp_executesql N'SELECT [Extent1].[OrderID] AS [OrderID], [Extent1].[CustomerID] AS [CustomerID], [Extent1].[SalespersonPersonID] AS [SalespersonPersonID], [Extent1].[PickedByPersonID] AS [PickedByPersonID], [Extent1].[ContactPersonID] AS [ContactPersonID], [Extent1].[BackorderOrderID] AS [BackorderOrderID], [Extent1].[OrderDate] AS [OrderDate], [Extent1].[ExpectedDeliveryDate] AS [ExpectedDeliveryDate], [Extent1].[CustomerPurchaseOrderNumber] AS [CustomerPurchaseOrderNumber], [Extent1].[IsUndersupplyBackordered] AS [IsUndersupplyBackordered], [Extent1].[Comments] AS [Comments], [Extent1].[DeliveryInstructions] AS [DeliveryInstructions], [Extent1].[InternalComments] AS [InternalComments], [Extent1].[PickingCompletedWhen] AS [PickingCompletedWhen], [Extent1].[LastEditedBy] AS [LastEditedBy], [Extent1].[LastEditedWhen] AS [LastEditedWhen] FROM [Sales].[Orders] AS [Extent1] WHERE [Extent1].[CustomerID] = @EntityKeyValue1',N'@EntityKeyValue1 int',@EntityKeyValue1=1
  • 25. DATA MODIFICATION context.Customers .Where(t => t.Name == “Petya”) .Update(t => new Customer { Name = “Petya” }); PM> Install-Package EntityFramework.Extended context.BulkInsert(customers); PM> Install-Package Z.EntityFramework.Extensions
  • 26. OOP public class WideWorldImporters : DbContext { public WideWorldImporters(string conString) : base(conString) { } // Some DbSets }
  • 27. OOP public class WideWorldImporters : DbContext { public WideWorldImporters(string conString) : base(conString) { } public WideWorldImporters() : this("WideWorldImporters") { } // Some DbSets } using (var context = new WideWorldImport ers("TestConString")) { context.Orders.ToList(); }
  • 28. IF ORM IS SO BAD, WHAT’S SOLUTION? Micro-ORM • Handcraft SQL • Handle mappings • Less code (less functionality)
  • 29. DAPPER • Created for Stack Overflow • Very. VERY fast • Works with POCO and dynamics using (var connection = new SqlConnection(conString)) { var users = connection.Query<User>("select * from Users") .ToList(); } using (var connection = new SqlConnection(conString)) { IEnumerable<dynamic> users = connection.Query ("select * from Users") .ToList(); }
  • 30. DAPPER. MAPPING var sql = @"select * from Posts p left join Users u on u.Id = p.OwnerId order by p.Id"; var data = connection.Query<Post, User, Post>( sql, (post, user) => { post.Owner = user; return post;}); var posts = data.ToList();
  • 31. DAPPER. MULTI RESULTS var sql = @" select * from Customers where CustomerId = @id select * from Orders where CustomerId = @id select * from Returns where CustomerId = @id"; using (var multi = connection.QueryMultiple(sql, new {id=selectedId})) { var customer = multi.Read<Customer>().Single(); var orders = multi.Read<Order>().ToList(); var returns = multi.Read<Return>().ToList(); }
  • 32. DAPPER. PROCEDURES var user = cnn.Query<User>( "sp_GetUser", new {Id = 1}, commandType: CommandType.StoredProcedure) .SingleOrDefault();
  • 33. DAPPER. OUTPUT PARAMETERS var p = new DynamicParameters(); p.Add("@a", 11); p.Add("@b", dbType: DbType.Int32, direction: ParameterDirection.Output); p.Add("@c", dbType: DbType.Int32, direction: ParameterDirection.ReturnValue); con.Execute( "spMagicProc", p, commandType: CommandType.StoredProcedure); int b = p.Get<int>("@b"); int c = p.Get<int>("@c");
  • 34. PETAPOCO Fast like Dapper Works with strictly undecorated POCOs, or attributed almost-POCOs var user1 = db.Single<User>(1); var user2 = db.Single<User>("WHERE Name = @0", “Vasya");
  • 35. PETAPOCO var u = new User(); u.Name = “Vasya”; db.Insert(u); // Update it u.Name = “Kolya"; db.Update(u); // Delete it db.Delete(u); db.Delete<User>( "WHERE Name LIKE @0", “V%"); db.Update<User>( "SET Name=@0 WHERE Id>@1", “XXX", 10);
  • 36. SIMPLE DATA Fun project of @markrendle Seems to be dead But still is fun return Database.Open() .Users.FindAllByName(name) .FirstOrDefault();
  • 37. PERFORMANCE SELECT MAPPING OVER 500 ITERATIONS - POCO SERIALIZATION Method Duration Hand coded (using a SqlDataReader) 47ms Dapper ExecuteMapperQuery 49ms PetaPoco 52ms NHibernate SQL 104ms Linq 2 SQL ExecuteQuery 181ms Entity framework ExecuteStoreQuery 631ms SELECT MAPPING OVER 500 ITERATIONS - DYNAMIC SERIALIZATION Method Duration Dapper ExecuteMapperQuery (dynamic) 48ms Simple.Data 95ms
  • 38. CONCLUSIONS • ORM good or bad? Depends. • Micro-ORM is good alternative in cases when ORM is bad. • Knowing internal implementations or behavior can increase performance of using of the frameworks. • You should always understand at least one layer below what you are coding.