{"id":4609,"date":"2019-06-29T20:27:54","date_gmt":"2019-06-30T01:27:54","guid":{"rendered":"http:\/\/neosmart.net\/blog\/?p=4609"},"modified":"2019-07-01T16:25:38","modified_gmt":"2019-07-01T21:25:38","slug":"sqlite-cache-for-asp-net-core","status":"publish","type":"post","link":"https:\/\/neosmart.net\/blog\/sqlite-cache-for-asp-net-core\/","title":{"rendered":"A persistent cache for ASP.NET Core"},"content":{"rendered":"<p>One of the nicest things about ASP.NET Core is the availability of certain singleton models that greatly simplify some very common developer needs. Given that (depending on who you ask) one of the two hardest problems in computing is caching<sup id=\"rf1-4609\"><a href=\"#fn1-4609\" title=\"Although that *probably* refers more to cache coherence rather than simply key-value persistence, to be perfectly frank.\" rel=\"footnote\">1<\/a><\/sup>, it&#8217;s extremely helpful that ASP.NET Core ships with several models for caching data, chief of which are <code>IMemoryCache<\/code> and <code>IDistributedCache<\/code>, added to an ASP.NET Core application via dependency injection and then available to both the framework and the application itself. Although these two expose almost identical APIs, they differ rather significantly in semantics.<sup id=\"rf2-4609\"><a href=\"#fn2-4609\" title=\"It is extremely refreshing to see Microsoft adopting the Haskell\/Rust approach of using types to express\/convey intention and semantics rather than merely shape.\" rel=\"footnote\">2<\/a><\/sup><\/p>\n<p><!--more--><\/p>\n<p>As the name suggests, <code>IMemoryCache<\/code> is an in-memory cache with ephemeral storage: its contents are lost when the application is restarted. As for <code>IDistributedCache<\/code>, it&#8217;s a little confusing because the two are not actually counterparts: the opposite of an <code>IMemoryCache<\/code> would be a hypothetical <code>IPersistentCache<\/code>, whereas the primary motivation behind the existence of <code>IDistributedCache<\/code> is not to add persistence to <code>IMemoryCache<\/code> but rather turn it into an ostensibly shared cache accessible to multiple instances of an application sitting behind a load balancer (hopefully coherently so), and usually not on the same machine.<\/p>\n<p>As such, <code>IDistributedCache<\/code> does not officially have a persistence model&#8212;however by virtue of its out-of-process implementation, it somewhat guarantees key-value caching resilient to application or worker process recycling. As a result, both application developers and Microsoft themselves have taken to using it for persistence-related reasons as well.<sup id=\"rf3-4609\"><a href=\"#fn3-4609\" title=\"e.g. when no other secure storage can be found for data protection keys, the framework will try to store them in the IDistributedCache if it is available, otherwise they are regenerated at startup and the ability to decrypt secrets encrypted prior to the application restart is lost.\" rel=\"footnote\">3<\/a><\/sup> The problem is that the default implementations for <code>IDistributedCache<\/code> are focused more on the &#8220;distributed&#8221; part rather than the &#8220;persistent&#8221; part, and a) don&#8217;t necessarily guarantee persistence, and b) aren&#8217;t necessarily intended to be used in non-distributed scenarios.<\/p>\n<p>For production purposes, setting up the backend service fulfilling <code>IDistributedCache<\/code> (e.g. redis, NCache, or SQL Server) is generally easy enough for any competent sysadmin, but during development and testing developers may find themselves without any form of persistent caching, as the local <code>IMemoryCache<\/code> is lost at application startup but there may be no viable <code>IDistributedCache<\/code> offering readily available on the testing machine. Also keep in mind that unlike many alternatives, ASP.NET Core is a zero-dependency framework and doesn&#8217;t require background services, docker containers, etc. &#8212; not even a web server or IIS Express &#8212; and as a result many projects have no &#8220;configure a development environment&#8221; step apart from &#8220;open the solution file,&#8221; but to properly test components that rely on an <code>IDistributedCache<\/code> implementation to provide persistence, a developer has hitherto needed to install and configure 3rd party dependencies.<\/p>\n<p><code>NeoSmart.Caching.Sqlite<\/code> is <a href=\"https:\/\/github.com\/neosmart\/AspSqliteCache\" rel=\"nofollow\">an open source project<\/a> available as <a href=\"https:\/\/www.nuget.org\/packages\/NeoSmart.Caching.Sqlite\" rel=\"follow\">a nuget package<\/a> for .NET Standard 2.0 and in particular for ASP.NET Core 2.2 implementing <code>IDistributedCache<\/code> intended for single-machine deployment purposes (i.e. it is not actually a distributed cache but rather a persistent one), including both prototyping\/development and final production deployment. It leverages the in-process SQLite database to provide persistent-to-disk key-value caching, and (unlike the other <code>IDistributedCache<\/code> offerings) requires no installation, no background services, external network servers, or admin privileges to configure and deploy during both testing or production.<\/p>\n<pre><code class=\"language-csharp\">\/\/ using NeoSmart.Caching.Sqlite;\r\n\r\npublic void ConfigureServices(IServiceCollection services)\r\n{\r\n    ...\r\n\r\n    \/\/ Note: this *must* come before services.AddMvc()!\r\n    services.AddSqliteCache(options =&gt; {\r\n        options.CachePath = @\"C:\\data\\bazaar\\cache.db\";\r\n    });\r\n\r\n    services.AddMvc();\r\n\r\n    ...\r\n}\r\n<\/code><\/pre>\n<p>The closest option to a single-machine persistent cache available to developers\/sysadmins for .NET Core has probably been Microsoft SQL Express (via <code>Microsoft.Caching.SqlServer<\/code> with a <code>LocalDB<\/code> target configured in the connection string, but that has some considerable drawbacks:<\/p>\n<ul>\n<li>It required the separate installation (and maintenance, upgrades, security patching, etc) of Microsoft SQL Express,<\/li>\n<li>It required administrator privileges to initially install,<\/li>\n<li>It is almost impossible to deploy <code>LocalDB<\/code> in production, as the cache database should be pre-created at the command line but is user-specific, and by default IIS runs application pools in a restricted account with its own profile (and hence, its own <code>LocalDB<\/code> instances). Microsoft has indicated that localdb is not really intended for production use with IIS-hosted ASP.NET Core applications.<\/li>\n<\/ul>\n<p>The redis editions of <code>IDistributedCache<\/code> (via <code>Microsoft.Caching.Redis<\/code> and <code>Microsoft.Caching.StackExchangeRedis<\/code>) are also extremely popular, but have their own list of drawbacks, including<\/p>\n<ul>\n<li>There is no redis distribution available for Windows,<sup id=\"rf4-4609\"><a href=\"#fn4-4609\" title=\"Microsoft at one point had released their own fork but that is no longer maintained nor supported, and other community-initiated forks are available but are insanely out-of-date and unpatched.\" rel=\"footnote\">4<\/a><\/sup><\/li>\n<li>Developers wanting to use redis anyway would need to deploy a docker container to run the redis instance in the background to which they intend to connect to, or would otherwise have to configure an actual redis instance on a (possibly virtual) machine on the local network,<\/li>\n<li>The network configuration would necessarily defer from developer to developer, or else developers would be using shared redis instances for testing during development, leading to a headache either way.<\/li>\n<li>Finally, data persistence with redis is not very straightforward and depending on the version you have running may not be an option or else might need to be configured. (For things like data protection keys, not having guaranteed persistence semantics is a big deal breaker!)<\/li>\n<\/ul>\n<p>Third party offerings such as the <code>NCache<\/code> offerings suffer from similar drawbacks to SQL Server offering above, except they don&#8217;t even have the benefit of the <code>LocalDB<\/code> option whereby the constantly running background service may be avoided, while still needing to install, update, and maintain a system-wide background service and requiring admin privileges to boot.<\/p>\n<p><code>NeoSmart.Caching.Sqlite<\/code> suffers from none of these issues, and (as shown above) couldn&#8217;t be easier to use. What&#8217;s more,<\/p>\n<ul>\n<li>It&#8217;s implemented as an on-disk SQLite database, automatically created at first run (i.e. much easier to use than <code>Microsoft.Caching.SqlServer<\/code> with <code>LocalDB<\/code>, which necessitates manual initialization of the datastore), to which all keys and values are persisted.<\/li>\n<li>It doesn&#8217;t interop with the GC except when a cached value is retrieved and read into a managed object, so it suffers from none of the drawbacks that the default <code>IMemoryCache<\/code> suffers from with long lifetimes and\/or large objects leading to forced GC gen 2 collections.<\/li>\n<li><code>NeoSmart.Caching.Sqlite<\/code> and all its dependencies are automatically installed and upgraded along with the rest of your project&#8217;s dependencies via NuGet, Paket, or whatever else you&#8217;re already using: there&#8217;s no background service that needs to separately installed and kept patched and up-to-date, no admin privileges are need to deploy it.<\/li>\n<li>As such, your entire project can be deployed via MS Web Deploy without needing to install any third-party dependencies on the web server, etc.<\/li>\n<li><code>NeoSmart.Sqlite.Caching<\/code> is fully parallelized with non-blocking reads and writes, thanks to all the hard work the SQLite project has put into creating thread-safe multi-reader\/writer coherence out-of-the-box. On the .NET side of things, command readers are pre-created and pooled (and sometimes even lock-free to retrieve, thanks to the <code>ConcurrentBag<\/code>-backed command pool).<\/li>\n<\/ul>\n<p>To install via NuGet, add a package reference to <code>NeoSmart.Caching.Sqlite<\/code> or execute the following in the Package Manager Console:<\/p>\n<pre><code>Install-Package NeoSmart.Caching.Sqlite\r\n<\/code><\/pre>\n<p><code>NeoSmart.Caching.Sqlite<\/code> requires only that a rw-accessible path be specified where the database will be stored (it&#8217;ll be created if it doesn&#8217;t exist) and everything else is configured and deployed for you. After calling <code>services.AddSqliteCache(...)<\/code> in your <code>ConfigureServices()<\/code> method, the global <code>SqliteCache<\/code> instance becomes available via dependency injection as both the specific <code>SqliteCache<\/code> type and the abstract <code>IDistributedCache<\/code> type.<\/p>\n<p><strong>Note that <code>services.AddSqliteCache(...)<\/code> must come before the call to <code>services.AddMvc()<\/code>,<\/strong> as Microsoft has simplified a lot of code within the framework by instantiating an instance of <code>IMemoryCache<\/code> as a psuedo-<code>IDistributedCache<\/code> if no <code>IDistributedCache<\/code> is available.<sup id=\"rf5-4609\"><a href=\"#fn5-4609\" title=\"This lets code thereafter load IDistributedCache via dependency injection and results in the correct cache being used regardless of whether the backing store is memory for a single PC or an external distributed cache for shared instances.\" rel=\"footnote\">5<\/a><\/sup> This is done within the call to <code>services.AddMvc(...)<\/code> and will result in the wrong <code>IDistributedCache<\/code> being returned if <code>services.AddSqliteCache(...)<\/code> wasn&#8217;t called first!<\/p>\n<p>The NeoSmart ASP.NET Core SQLite cache fully implements the <code>IDistributedCache<\/code> interface, including both synchronous and asynchronous insert\/get\/remove\/refresh operations as well as sliding renewal on retrieve. Expired items are removed periodically in the background,<sup id=\"rf6-4609\"><a href=\"#fn6-4609\" title=\"The default interval can be changed or the background cleanup can be disabled altogether via the SqliteCacheOptions flavor of IOptions, configured in the call to services.AddSqliteCache(...).\" rel=\"footnote\">6<\/a><\/sup> and won&#8217;t accumulate indefinitely.<\/p>\n<p>The SQLite caching library for ASP.NET Core is released to the general public under the terms of the MIT open source license in hopes that it is useful and of some benefit. You can <a href=\"https:\/\/github.com\/neosmart\/AspSqliteCache\" rel=\"nofollow\">find it on GitHub<\/a> to star and save, where you can also file any issues or pull requests. Improvements to the library or accompanying documentation are welcome.<\/p>\n<p><a href=\"https:\/\/www.nuget.org\/packages\/NeoSmart.Caching.Sqlite\" rel=\"follow\"><img loading=\"lazy\" decoding=\"async\" class=\"no-border alignnone wp-image-4623 colorbox-4609\" src=\"https:\/\/neosmart.net\/blog\/wp-content\/uploads\/2019\/06\/Get-from-nuget.png\" alt=\"\" width=\"201\" height=\"60\" \/><\/a>\u00a0\u00a0\u00a0<a href=\"https:\/\/github.com\/neosmart\/AspSqliteCache\" rel=\"nofollow\"><img loading=\"lazy\" decoding=\"async\" class=\"no-border alignnone wp-image-4622 colorbox-4609\" src=\"https:\/\/neosmart.net\/blog\/wp-content\/uploads\/2019\/06\/Explore-on-GitHub.png\" alt=\"\" width=\"201\" height=\"60\" \/><\/a><\/p>\n<hr \/>\n<div class=\"sendy_widget\" style='margin-bottom: 0.5em;'>\n<p><em>If you would like to receive a notification the next time we release a nuget package for .NET or release resources for .NET Core and ASP.NET Core, you can subscribe below. Note that you'll only get notifications relevant to .NET programming and development by NeoSmart Technologies. If you want to receive email updates for all NeoSmart Technologies posts and releases, please sign up in the sidebar to the right instead.<\/em><\/p>\n<iframe tabIndex=-1 onfocus=\"sendy_no_focus\" src=\"https:\/\/neosmart.net\/sendy\/subscription?f=BUopX8f2VyLSOb892VIx6W4BB8V5K2ReYGLVwsfKUZLXCc892Ffz8rIgRyIGoE22cZVr&title=Join+the+dotnet+mailing+list\" style=\"height: 300px; width: 100%;\"><\/iframe>\n<\/div>\n<script type=\"text\/javascript\">function sendy_no_focus(e) { e.preventDefault(); }<\/script>\n<hr class=\"footnotes\"><ol class=\"footnotes\"><li id=\"fn1-4609\"><p>Although that *probably* refers more to cache coherence rather than simply key-value persistence, to be perfectly frank.&nbsp;<a href=\"#rf1-4609\" class=\"backlink\" title=\"Jump back to footnote 1 in the text.\">&#8617;<\/a><\/p><\/li><li id=\"fn2-4609\"><p>It is extremely refreshing to see Microsoft adopting the Haskell\/Rust approach of using types to express\/convey intention and semantics rather than merely shape.&nbsp;<a href=\"#rf2-4609\" class=\"backlink\" title=\"Jump back to footnote 2 in the text.\">&#8617;<\/a><\/p><\/li><li id=\"fn3-4609\"><p>e.g. when no other secure storage can be found for data protection keys, the framework will try to store them in the <code>IDistributedCache<\/code> if it is available, otherwise they are regenerated at startup and the ability to decrypt secrets encrypted prior to the application restart is lost.&nbsp;<a href=\"#rf3-4609\" class=\"backlink\" title=\"Jump back to footnote 3 in the text.\">&#8617;<\/a><\/p><\/li><li id=\"fn4-4609\"><p>Microsoft at one point had released their own fork but that is no longer maintained nor supported, and other community-initiated forks are available but are insanely out-of-date and unpatched.&nbsp;<a href=\"#rf4-4609\" class=\"backlink\" title=\"Jump back to footnote 4 in the text.\">&#8617;<\/a><\/p><\/li><li id=\"fn5-4609\"><p>This lets code thereafter load <code>IDistributedCache<\/code> via dependency injection and results in the correct cache being used regardless of whether the backing store is memory for a single PC or an external distributed cache for shared instances.&nbsp;<a href=\"#rf5-4609\" class=\"backlink\" title=\"Jump back to footnote 5 in the text.\">&#8617;<\/a><\/p><\/li><li id=\"fn6-4609\"><p>The default interval can be changed or the background cleanup can be disabled altogether via the <code>SqliteCacheOptions<\/code> flavor of <code>IOptions<\/code>, configured in the call to <code>services.AddSqliteCache(...)<\/code>.&nbsp;<a href=\"#rf6-4609\" class=\"backlink\" title=\"Jump back to footnote 6 in the text.\">&#8617;<\/a><\/p><\/li><\/ol>","protected":false},"excerpt":{"rendered":"<p>One of the nicest things about ASP.NET Core is the availability of certain singleton models that greatly simplify some very common developer needs. Given that (depending on who you ask) one of the two hardest problems in computing is caching1, &hellip; <a href=\"https:\/\/neosmart.net\/blog\/sqlite-cache-for-asp-net-core\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":505,"featured_media":4447,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[997,190,963,52,996],"class_list":["post-4609","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-software","tag-asp-net-core","tag-caching","tag-nuget","tag-open-source","tag-sqlite"],"aioseo_notices":[],"jetpack_featured_media_url":"https:\/\/neosmart.net\/blog\/wp-content\/uploads\/cropped-NeoSmart-Padded.png","jetpack_shortlink":"https:\/\/wp.me\/p4xDa-1cl","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/posts\/4609","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/users\/505"}],"replies":[{"embeddable":true,"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/comments?post=4609"}],"version-history":[{"count":26,"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/posts\/4609\/revisions"}],"predecessor-version":[{"id":4638,"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/posts\/4609\/revisions\/4638"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/media\/4447"}],"wp:attachment":[{"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/media?parent=4609"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/categories?post=4609"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/neosmart.net\/blog\/wp-json\/wp\/v2\/tags?post=4609"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}