Jekyll2021-01-29T17:00:34+01:00https://www.claasloader.de/feed.xmlclaasloaderMy IT blogClaas ZurawskiDiff Excel Files2021-01-29T16:21:00+01:002021-01-29T16:21:00+01:00https://www.claasloader.de/excel/programming/2021/01/29/diff-two-excel-files<p>As developers, we are used to see the difference between two (text) files via <code class="highlighter-rouge">diff</code>, <code class="highlighter-rouge">git diff</code>, or related tools. How to see the difference between two Excel files?</p>
<p>There are many options, and your choice would probably depend on what you mean by “difference”.</p>
<h2 id="1-csv--meld">1) CSV + Meld</h2>
<p>This is the easiest approach. Save both files as <code class="highlighter-rouge">.csv</code>, and use <code class="highlighter-rouge">diff</code> to compare them. Alternatively, you can use <code class="highlighter-rouge">Meld</code>, which is basically a UI for <code class="highlighter-rouge">diff</code>.</p>
<p>For big Excel files, or for two files with lots of differences, it’s usage is limited.</p>
<h2 id="2-excelcompare">2) ExcelCompare</h2>
<p><a href="https://github.com/na-ka-na/ExcelCompare">ExcelCompare</a> is a sophisticated Java based tool with lots of options. I guess it’s most useful if you already have an idea what kind of difference to expect.</p>
<h2 id="3-csv--mysql">3) CSV + MySQL</h2>
<p>This approach requires the most effort but comes with high flexibility.</p>
<p>First, save both Excel files in <code class="highlighter-rouge">.csv</code> format.</p>
<p>Then, create a scratch database in MySQL:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>> create database excel_diff_tmp;
</code></pre></div></div>
<p>Next, create two tables which match your Excel file’s tables:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CREATE TABLE `sheet1` (
id int(11) NOT NULL AUTO_INCREMENT,
column_a varchar(255) DEFAULT NULL,
column_b varchar(255) DEFAULT NULL,
# ...
PRIMARY KEY (`id`)
) DEFAULT CHARSET=utf8;
</code></pre></div></div>
<p>Now, load both <code class="highlighter-rouge">.csv</code> files into the database using <code class="highlighter-rouge">LOAD DATA LOCAL INFILE</code>:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>LOAD DATA LOCAL INFILE '/path/to/file1.csv'
INTO TABLE sheet1
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
</code></pre></div></div>
<p>You now have the full power of SQL at you hands to compare these two tables. You can do things like</p>
<ul>
<li>Show rows which are in file 1, but not in file 2 (using all columns)</li>
<li>Show rows which are in file 1, but not in file 2 (using only some columns)</li>
<li>Show rows equal in both files</li>
<li>etc.</li>
</ul>
<p>From MySQL Workbench, you could export the result back as a <code class="highlighter-rouge">.csv</code> file, which you can finally convert to Excel format.</p>Claas ZurawskiAs developers, we are used to see the difference between two (text) files via diff, git diff, or related tools. How to see the difference between two Excel files?Rails Deployment in 20202020-09-04T17:00:00+02:002020-09-04T17:00:00+02:00https://www.claasloader.de/ruby/rails/devops/programming/2020/09/04/rails-deployment-in-2020<p>In 2019, <a href="https://www.wintermeyer-consulting.de/">Stefan Wintermeyer</a> wrote a blog post <a href="https://medium.com/@wintermeyer/https-medium-com-wintermeyer-rails-needs-active-deployment-65c207858c3">Rails needs Active Deployment</a> which received some attention in the Rails community. Later on, he appeared as a guest on the <a href="https://dev.to/rubyrogues/mrs-083-stefan-wintermeyer">Ruby Rogues Podcast #83</a> explaining his idea.</p>
<p>His main point is that deployment is (too) difficult for Rails applications, particularly for new developers. And opposed to other aspects of the web development process, Rails doesn’t provide you anything to make your life easier.</p>
<p>This drew my interest, and I’ve investigated a little bit what’s the situation for Rails Deployment in 2020. Here’s what I’ve found.</p>
<h2 id="a-little-bit-of-history">A Little Bit of History</h2>
<p>For quite a long time, the Rails team particularly promoted Rails as being easy to deploy. The <a href="https://rubyonrails.org/">Rails website</a> showed a <a href="https://web.archive.org/web/20090802034435/http://rubyonrails.org/deploy">Deploy</a> page where it claimed</p>
<blockquote>
<p>Deploying Ruby on Rails is easy</p>
</blockquote>
<p>and below</p>
<blockquote>
<p>Thanks to […], launching is now almost as easy as developing</p>
</blockquote>
<p>At the same time, the most popular book about Rails was <a href="https://pragprog.com/book/rails6/agile-web-development-with-rails-6">“Agile Web Development with Rails”</a> by Sam Ruby et al. There, it says</p>
<blockquote>
<p>Ruby on Rails is a framework that makes it easier to develop, deploy, and
maintain web applications (3rd ed., page 14)</p>
</blockquote>
<p>At that time, the standard way to deploy a Rails app was <a href="https://capistranorb.com/">Capistrano</a> + <a href="https://www.phusionpassenger.com/">Passenger</a>. Typically, you would deploy your app to a Linux server (physical or VM).</p>
<p>In 2016, the “Deploy” page was removed from the Rails website without any replacement.</p>
<p>In 2017, the Rails 5 edition of “Agile Web Development with Rails” was released. It still contained the chapter about deploying with Capistrano + Passenger, but it was enhanced by a chapter about deploying to <a href="https://www.heroku.com/">Heroku</a>.</p>
<p>In 2019, <a href="https://github.com/rails/rails/commit/2584762cd9876780929098ad847ccf53aa45c10e">Capistrano was removed from the default Gemfile</a>. DHH’s commit message said</p>
<blockquote>
<p>Capistrano is no longer a dominant force in the deployment strategy for new apps</p>
</blockquote>
<p>In 2020, the Rails 6 edition of the book was released. The deployment chapter was removed.</p>
<h2 id="situation-in-2020-deployment-out-of-scope">Situation in 2020: Deployment “out of scope”</h2>
<p>So checking official Rails documentation right now, there’s almost nothing about deployment, and it looks like this is by intention. Searching Github issues and the Rails forum shows two statements from core developers (both from 2018):</p>
<blockquote>
<p>We prefer not to cover such practices in the Rails documentation, because the majority of what you have to say depends on external software, web servers, application servers, their configuration, databases, operating systems, hosting providers, etc.</p>
</blockquote>
<p><a href="https://discuss.rubyonrails.org/t/add-guide-to-deploy-rails-applications/73282">https://discuss.rubyonrails.org/t/add-guide-to-deploy-rails-applications/73282</a></p>
<blockquote>
<p>I’m in general -1 about covering this kind of content in the Rails docs. While related, I see it as being out of scope.</p>
</blockquote>
<p><a href="https://github.com/rails/rails/issues/32568#issuecomment-381319480">https://github.com/rails/rails/issues/32568#issuecomment-381319480</a></p>
<p>While of course you can argue that the whole deployment process is somehow “unrelated” to the core framework documentation, most Rails <a href="https://hotframeworks.com/">competitors</a> do provide deployment guidelines:</p>
<ul>
<li><a href="https://docs.spring.io/spring-boot/docs/2.2.2.RELEASE/reference/htmlsingle/#deployment">Spring Boot</a></li>
<li><a href="https://docs.djangoproject.com/en/2.2/howto/deployment/wsgi/modwsgi/">Django</a></li>
<li><a href="http://expressjs.com/en/guide/behind-proxies.html">Express</a></li>
<li><a href="https://laravel.com/docs/5.8">Laravel</a></li>
<li><a href="https://www.playframework.com/documentation/2.7.x/Deploying">Play</a></li>
<li><a href="http://flask.pocoo.org/docs/1.0/deploying/">Flask</a></li>
<li><a href="https://dotnet.microsoft.com/learn/aspnet">ASP.NET</a></li>
</ul>
<h2 id="dhh-well-explore-more-in-this-space-2019">DHH: “We’ll explore more in this space” (2019)</h2>
<p>In January 2019, DHH <a href="https://discuss.rubyonrails.org/t/discussion-rails-needs-active-deployment/73443">posted in the official Rails forum</a></p>
<blockquote>
<p>I’m interested in exploring better defaults for deployment going forward. Many of the fundamentals have improved dramatically, whether they be simply Docker, Kubernetes, cloud providers at large, or Heroku. So we’ll explore more in this space, but there’s nothing concrete at the moment.</p>
</blockquote>
<p>To my knowledge, nothing has happened so far.</p>
<h2 id="the-docker-book">The Docker Book</h2>
<p>In recent years, Docker has become one of the industry standards for running web applications. So I was happy that a book about the combination of Rails and Docker was released: <a href="https://pragprog.com/titles/ridocker/docker-for-rails-developers/">“Docker for Rails Developers”</a> by Rob Isenberg (2019). This book is also often recommended in forums when Rails deployment is discussed.</p>
<p>However, while this is a good book about Docker itself, it doesn’t provide you an (easy) guideline how to deploy a Rails app to a Docker environment. Additionally, having to read 200 pages to get a basic deployment workflow is too much, in my opinion. (If you need a more advanced solution with clusters etc., this book might be helpful, though.)</p>
<h2 id="final-thoughts">Final Thoughts</h2>
<p>In a <a href="https://martinfowler.com/bliki/DevOpsCulture.html">DevOps</a> environment, developers have to care about deployment, and I guess for a lot of Rails developers this has always been the case. When I started with Rails (in 2010), I was astonished how easy it was to deploy Rails to production, just following a short chapter of “Agile Web Development with Rails”. 10 years later, situation has become worse, not better. Deployment documentation has been removed from official resources, without replacments. You have to rely on external documentation, forums, StackOverflow, or commercial products now.</p>
<h2 id="appendix-some-resources">Appendix: Some Resources</h2>
<p>If you came to this post to get an overview about deployment options, here’s a short (very incomplete) list of resources:</p>
<ul>
<li><a href="https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/apache/oss/install_language_runtime.html">Apache + Passenger</a></li>
<li><a href="https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/nginx/oss/install_language_runtime.html">Nginx + Passenger</a></li>
<li><a href="https://github.com/puma/puma/blob/master/docs/deployment.md">Puma</a></li>
<li><a href="https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ruby-rails-tutorial.html">AWS Elastic Beanstalk</a></li>
<li><a href="https://cloud.google.com/ruby/rails">Google App Engine</a></li>
<li><a href="https://www.hatchbox.io/">Hatchbox</a></li>
<li><a href="https://capistranorb.com/">Capistrano</a></li>
<li><a href="https://github.com/seattlerb/vlad">Vlad</a></li>
<li><a href="https://github.com/mina-deploy/mina">Mina</a></li>
<li><a href="https://www.heroku.com/">Heroku</a></li>
<li><a href="https://render.com/">Render</a></li>
<li><a href="https://www.digitalocean.com/community/tutorials/how-to-install-ruby-on-rails-with-rbenv-on-ubuntu-18-04">DigitalOcean </a></li>
<li><a href="https://docs.docker.com/compose/rails/">Docker + Puma</a></li>
</ul>Claas ZurawskiIn 2019, Stefan Wintermeyer wrote a blog post Rails needs Active Deployment which received some attention in the Rails community. Later on, he appeared as a guest on the Ruby Rogues Podcast #83 explaining his idea.Git Commit Best Practices2020-02-26T17:00:00+01:002020-02-26T17:00:00+01:00https://www.claasloader.de/git/programming/2020/02/26/git-commit-best-practices<p>Even though everybody uses git nowadays, people sometimes struggle <em>how</em> to use it efficiently. As <code class="highlighter-rouge">git commit</code> it the most important git command, let me present some guidelines which we found useful.</p>
<h2 id="1-commit-frequency">1) Commit frequency</h2>
<p>How often should you commit? There’s a simple rule:</p>
<p><strong>Commit early and often</strong></p>
<p>Why? There are several reasons for that:</p>
<h3 id="for-your-team">For your team</h3>
<p><strong>Visibility:</strong> If your work lives on your local machine only, nobody can be aware what you are currently working on. Neither could anybody pick up your task in case you get sick or you have to stop working on it for any other reason.</p>
<p><strong>Project / feature history:</strong> Many small commits with useful commit messages also work as some kind of documentation. Your team mates can get an impression of the work that has been done by looking at the git history.</p>
<h3 id="for-yourself">For yourself</h3>
<p><strong>Backup:</strong> In case you work on a task for several days without pushing to the central repository, you risk loosing your work in case your PC goes down (assuming you have no other backup system in place).</p>
<p><strong>Safety net:</strong> Let’s say you copy + past to the wrong editor window. How to recover? Sure all editors have some “undo” function, but often times it only works until the last time you hit “Save”. Early and often git commits give you a comfortable safety net wich allows you to always go back to the latest commit version in case of an accident (or maybe your brilliant refactoring idea turns out to be not so brilliant).</p>
<p><strong>Status:</strong> After pausing for some days/weeks, you will find it easier to remember what tasks have been done and what have not by looking at your git history.</p>
<h3 id="bad-practices">Bad practices</h3>
<ul>
<li>
<p>One single commit at the end of the day: Sometimes I saw people doing one big commit at the end of their work day. This ensures there’s a backup, but you would not have the other benefits mentioned above</p>
</li>
<li>
<p>Local commits without <code class="highlighter-rouge">git push</code>: There’s actually no reason why a commit should not be pushed. If you have commits which would put the application in “not ready yet” state, use <a href="https://www.martinfowler.com/bliki/FeatureBranch.html">feature branches</a>, or – preferrably – <a href="https://www.martinfowler.com/articles/feature-toggles.html">feature toggles</a>.</p>
</li>
</ul>
<h2 id="2-how-to-group-changes-into-commits">2) How to group changes into commits</h2>
<p>If you pay a little attention how commits are grouped, your commit history becomes easy to read and can even serve as some kind of documentation. To achieve this, it’s important to <em>group related changes together</em>.</p>
<ul>
<li>
<p>Changes for a bugfix</p>
</li>
<li>
<p>Changes for a new feature</p>
</li>
<li>
<p>Refactoring</p>
</li>
<li>
<p>Changes triggered by a linter (code conventions / formattings)</p>
</li>
<li>
<p>Whitespace changes</p>
</li>
</ul>
<p>It’s quite helpful to separate refactoring changes from feature development changes. When browsing through the commit history later on, it’s much easier to understand why a particular change was made.</p>
<p>Similar idea for whitespace and code formatting changes. They are usually less interesting and can be easily skipped if they happen in a separate commit. On the other hand, if you group code formatting changes and feature changes together, the diff becomes harder to understand.</p>
<p>The worst is actually grouping changes for two or more unrelated features together. The git diff would become quite big, and you would have to read it line by line to understand which change belongs to which feature.</p>
<p>The Linux kernel project has <a href="https://www.kernel.org/doc/html/latest/process/submitting-patches.html#separate-your-changes">similar guidelines</a>.</p>
<h2 id="3-when-to-commit">3) When to commit</h2>
<p>While you should commit early and often, you should commit <em>when it makes sense</em>. I’ve seen people abusing version control as a backup solution by committing every x minutes (or hours), regardless of the current state. Sometimes also the question comes up whether committing should be allowed while the code includes known bugs or event compile errors. From my point of view, this is a bad idea. When somebody checks out the “bad” version later on, it’s unclear whether there’s a real bug or the code is just in an “in progress” state. Therefore, we always set these guidelines: whenever you commit,</p>
<ul>
<li>
<p>there should be no compilation errors</p>
</li>
<li>
<p>tests should pass (unit tests etc.)</p>
</li>
<li>
<p>linters and other code analyzers should not complain</p>
</li>
</ul>
<p>On the other hand, your feature doesn’t have to be complete (see above).</p>
<h2 id="4-commit-messages">4) Commit messages</h2>
<p>We prefer commit messages of one or two sentences (roughly, 15 - 100 characters), summarizing what has been changed. Some things you should consider:</p>
<ul>
<li>
<p>Most git hosting tools like GitHub, Bitbucket etc. have a “commits” page where all commits are listed, showing most recent commits first. There’s usually one row for the commit message, and it would display only the first 60 chars (something like that). So the key point of your commit message should already be included in the first 60 chars</p>
</li>
<li>
<p>While scanning throught the git history, you would usually read only what’s displayed on the “commits” page directly. You would not click on each commit to expand the full message</p>
</li>
<li>
<p>On the other hand, the commit message shouldn’t be to short neither. It should give the reader some good idea what the commit is about</p>
</li>
<li>
<p>It’s quite helpful to have a convention that bugfix commits always start with <code class="highlighter-rouge">Bugfix:</code>, refactorings always start with <code class="highlighter-rouge">Refactoring: ...</code> etc.</p>
</li>
<li>
<p>In case you use a ticket system (Jira, YouTrack, etc.), you should include the ticket number in the message</p>
</li>
<li>
<p>Make sure your message is informative. <code class="highlighter-rouge">Fixed weird bug</code> only tells that the commit is a bugfix, but which bug? Why is it “weird”?</p>
</li>
<li>
<p>Another bad practice is to repeat the code changes, e.g. <code class="highlighter-rouge">Added method 'write()' to class A</code>. These messages are very hard to understand, particularly if you use generic names like <code class="highlighter-rouge">write()</code>, <code class="highlighter-rouge">generate()</code>, <code class="highlighter-rouge">send()</code> etc. In this case, the commit message is more or less the same as the diff, which makes it less useful.</p>
</li>
</ul>
<h2 id="5-the-power-of-annotate">5) The power of annotate</h2>
<p>The recommendations given above would give you a nice history you can read through on GitHub’s / Bitbucket’s “commits” page. However, they would also leverage the power of another – often overlooked – git command: <code class="highlighter-rouge">git annotate</code>.</p>
<p>For a given file, annotate basically gives you the most recent commit information for each line. Most editors have some annotate command (GitHub and Bitbucket call it <code class="highlighter-rouge">blame</code>).</p>
<p>In case you followed the guidelines above, <code class="highlighter-rouge">annotate</code> would give you magically some kind of documentation for your source file. For each method, statement, you would be able to see:</p>
<ul>
<li><em>Who</em> changed the line the last time?</li>
<li><em>When</em> was it changed the last time?</li>
<li>What was the <em>reason</em> for the change? What was the <em>context</em> of the change?</li>
</ul>
<p>Particulary for legacy projects, the <code class="highlighter-rouge">annotate</code> view can be extremely helpful.</p>Claas ZurawskiEven though everybody uses git nowadays, people sometimes struggle how to use it efficiently. As git commit it the most important git command, let me present some guidelines which we found useful.Integrate Bitbucket Server 6 and Jenkins (without plugin)2020-01-07T17:00:00+01:002020-01-07T17:00:00+01:00https://www.claasloader.de/bitbucket/jenkins/devops/2020/01/07/integrate-bitbucket-server-and-jenkins<p>If you use <a href="https://bitbucket.org/product/enterprise">Bitbucket Server</a> to host your code and <a href="https://jenkins.io/">Jenkins</a> to run your tests, you have to link both systems. I.e., once you push changes to Bitbucket, you want Jenkins to start a new build automatically.</p>
<p>For some reason, there are lots of different solutions for this task, but most of them are outdated or quite complicated. I’ll give a short overview about the different ways I’ve tried and show a really easy solution at the end.</p>
<h2 id="official-solution-bitbucket-server-68">Official solution (Bitbucket Server 6.8+)</h2>
<p>On the <a href="https://confluence.atlassian.com/bitbucketserver/bitbucket-server-6-8-release-notes-979409325.html#BitbucketServer6.8releasenotes-jenkinsSeamlesslyintegratewithJenkins">release notes for Bitbucket Server 6.8</a>, Atlassian links to a Jenkins <a href="https://plugins.jenkins.io/atlassian-bitbucket-server-integration">plugin</a>.</p>
<ul>
<li>The <a href="https://wiki.jenkins.io/display/JENKINS/Bitbucket+Server+integration+plugin+for+Jenkins">wiki page</a> marks it as “beta”</li>
<li>You have to create a (Bitbucket) personal access token with Admin rights for everything - unclear why this is needed</li>
<li>You have to create an “auth credential”, unclear what this means</li>
<li>During a quick test, I couldn’t make it work</li>
</ul>
<h2 id="jenkins-plugin-bitbucket">Jenkins plugin ‘Bitbucket’</h2>
<p>Lots of blog posts mention this <a href="https://plugins.jenkins.io/bitbucket">plugin</a>, it looks like it has been the common solution for a while. However, it’s unmaintained now.</p>
<ul>
<li>Unmaintained</li>
<li>Another plugin needed on Bitbucket side</li>
<li>A <a href="https://github.com/jenkinsci/bitbucket-plugin/pull/63">Github issue</a> mentions it would work with standard Webhooks; however it’s unclear to me how to do it</li>
<li>No documentation how to use it</li>
</ul>
<h2 id="jenkins-plugin-bitbucket-branch-source">Jenkins plugin ‘Bitbucket Branch Source’</h2>
<p>This <a href="https://wiki.jenkins.io/display/JENKINS/Bitbucket+Branch+Source+Plugin">plugin</a> is also mentioned sometimes. I haven’t tried it.</p>
<h2 id="easiest-solution-no-plugins-needed">Easiest solution (no plugins needed)</h2>
<p>Luckily, there’s no particular Bitbucket plugin needed at all. All we need is Jenkins’ basic <a href="https://plugins.jenkins.io/git">Git plugin</a>, but most probably, you have this already.</p>
<p>Using ‘polling’ would already work. There would be some delay between pushing and building - depending on your use case, this can be acceptable or not.</p>
<p>Additionally, the Git plugin supports a simple GET request to trigger a poll immediately:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>curl <span class="nt">-i</span> <span class="s2">"http://my.jenkins.url/git/notifyCommit?url=ssh://git@my.bitbucket.url:7999/myproject/myapp.git"</span></code></pre></figure>
<p>In Bitbucket, just create a new Webhook with a url like the one above, and you are done.</p>
<h3 id="resources">Resources</h3>
<ul>
<li><a href="https://plugins.jenkins.io/git#build-initiation-extensions">https://plugins.jenkins.io/git#build-initiation-extensions</a></li>
<li><a href="https://wiki.jenkins.io/display/JENKINS/Git+Plugin#GitPlugin-Pushnotificationfromrepository">https://wiki.jenkins.io/display/JENKINS/Git+Plugin#GitPlugin-Pushnotificationfromrepository</a></li>
</ul>Claas ZurawskiIf you use Bitbucket Server to host your code and Jenkins to run your tests, you have to link both systems. I.e., once you push changes to Bitbucket, you want Jenkins to start a new build automatically.Dead Easy Way to Profile a Rails App2019-08-27T17:00:00+02:002019-08-27T17:00:00+02:00https://www.claasloader.de/programming/ruby/rails/2019/08/27/dead-easy-way-to-profile-rails-app<p>While investigating performance problems in a Rails app, a <em>profiler</em> is quite helpful. <a href="https://github.com/tmm1/stackprof">stackprof</a> is such a tool which can be added to an existing Rails app quite easily.</p>
<h2 id="installation">Installation</h2>
<p>As usual, add</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gem 'stackprof'
</code></pre></div></div>
<p>to your <code class="highlighter-rouge">Gemfile</code> and run</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>bundle <span class="nb">install</span></code></pre></figure>
<p>To enable profiling, add this line to your configuration:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># development.rb</span>
config.middleware.use<span class="o">(</span>StackProf::Middleware,
out: <span class="s1">'tmp/stackprof.dump'</span>,
enabled: <span class="nb">true</span>,
mode: :wall,
interval: 1000,
save_every: 5<span class="o">)</span></code></pre></figure>
<p>Important here is the <code class="highlighter-rouge">save_every: 5</code> parameter. It means that profile data is written after 5 HTTP request to your application.</p>
<h2 id="usage">Usage</h2>
<p>Restart your server</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>rails s</code></pre></figure>
<p>And make 5 requests to your app (usually you would investigate performance issues for a single action, i.e. you would run the <em>same</em> request 5 times in a row).</p>
<p>Now, raw profile data should be created in files <code class="highlighter-rouge">tmp/stackprof-wall-*</code>.</p>
<p>Finally, you would generate profiling information by</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>stackprof tmp/stackprof-wall-<span class="k">*</span>dump</code></pre></figure>
<p>This gives you something like</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>stackprof tmp/stackprof-wall-<span class="k">*</span>dump
<span class="o">==================================</span>
Mode: wall<span class="o">(</span>1000<span class="o">)</span>
Samples: 20152 <span class="o">(</span>4.57% miss rate<span class="o">)</span>
GC: 590 <span class="o">(</span>2.93%<span class="o">)</span>
<span class="o">==================================</span>
TOTAL <span class="o">(</span>pct<span class="o">)</span> SAMPLES <span class="o">(</span>pct<span class="o">)</span> FRAME
15666 <span class="o">(</span>77.7%<span class="o">)</span> 15666 <span class="o">(</span>77.7%<span class="o">)</span> Puma::Single#run
590 <span class="o">(</span>2.9%<span class="o">)</span> 590 <span class="o">(</span>2.9%<span class="o">)</span> <span class="o">(</span>garbage collection<span class="o">)</span>
502 <span class="o">(</span>2.5%<span class="o">)</span> 436 <span class="o">(</span>2.2%<span class="o">)</span> Logger::LogDevice#write
265 <span class="o">(</span>1.3%<span class="o">)</span> 265 <span class="o">(</span>1.3%<span class="o">)</span> ActiveRecord::LogSubscriber#ignored_callstack
227 <span class="o">(</span>1.1%<span class="o">)</span> 227 <span class="o">(</span>1.1%<span class="o">)</span> Logger::Formatter#format_datetime
224 <span class="o">(</span>1.1%<span class="o">)</span> 224 <span class="o">(</span>1.1%<span class="o">)</span> ActiveRecord::Base.logger
157 <span class="o">(</span>0.8%<span class="o">)</span> 157 <span class="o">(</span>0.8%<span class="o">)</span> Mysql2::Client#query
119 <span class="o">(</span>0.6%<span class="o">)</span> 119 <span class="o">(</span>0.6%<span class="o">)</span> Concurrent::Collection::NonConcurrentMapBackend#[]
118 <span class="o">(</span>0.6%<span class="o">)</span> 118 <span class="o">(</span>0.6%<span class="o">)</span> Sprockets::Mime#compute_extname_map
97 <span class="o">(</span>0.5%<span class="o">)</span> 97 <span class="o">(</span>0.5%<span class="o">)</span> ActiveRecord::LogSubscriber#sql_color
358 <span class="o">(</span>1.8%<span class="o">)</span> 92 <span class="o">(</span>0.5%<span class="o">)</span> Logger::Formatter#call
1047 <span class="o">(</span>5.2%<span class="o">)</span> 83 <span class="o">(</span>0.4%<span class="o">)</span> Logger#add
63 <span class="o">(</span>0.3%<span class="o">)</span> 63 <span class="o">(</span>0.3%<span class="o">)</span> ActiveSupport::TaggedLogging::Formatter#current_tags
53 <span class="o">(</span>0.3%<span class="o">)</span> 53 <span class="o">(</span>0.3%<span class="o">)</span> ActiveRecord::Result#hash_rows
1156 <span class="o">(</span>5.7%<span class="o">)</span> 49 <span class="o">(</span>0.2%<span class="o">)</span> ActiveSupport::Logger#add
48 <span class="o">(</span>0.2%<span class="o">)</span> 48 <span class="o">(</span>0.2%<span class="o">)</span> ActiveRecord::QueryMethods#get_value</code></pre></figure>
<h2 id="wall-vs-cpu">wall vs cpu</h2>
<p>Depending on your use case, you might want to</p>
<ul>
<li>analyze time spent in your app <strong>together</strong> with other systems like a database or an external http service (<code class="highlighter-rouge">wall</code> mode = total runtime)</li>
</ul>
<p>– or –</p>
<ul>
<li>analyze time spent in your app <strong>itself</strong> (<code class="highlighter-rouge">cpu</code> mode = cpu time)</li>
</ul>
<p>You might have noticed that in the listing above, most time is spent for <code class="highlighter-rouge">Puma.single</code>. This is because we’ve used <code class="highlighter-rouge">mode: :wall</code> in our configuration. <a href="https://twitter.com/nateberkopec/status/1012376192450641920">Puma.single basically means i/o wait</a>.</p>
<h2 id="how-it-works">How it works</h2>
<p>Note that stackprof is a <em>sampling</em> profiler:</p>
<blockquote>
<p>Stackprof is a sampling call-stack profile for Ruby 2.1+.
Instead of tracking all method calls, it will simply collect the current stack trace of a running program at fixed intervals. Methods that appear on top of the stack trace most often, are the methods your program spends most of its time in.
(<a href="https://makandracards.com/makandra/38975-stackprof-sampling-call-stack-profiler-for-ruby">https://makandracards.com/makandra/38975-stackprof-sampling-call-stack-profiler-for-ruby</a>)</p>
</blockquote>
<h2 id="resources">Resources</h2>
<ul>
<li><a href="https://scoutapm.com/blog/profiling-rails-with-stackprof">https://scoutapm.com/blog/profiling-rails-with-stackprof</a></li>
<li><a href="https://samsaffron.com/archive/2018/01/18/my-production-ruby-on-rails-cpu-is-at-100-now-what">https://samsaffron.com/archive/2018/01/18/my-production-ruby-on-rails-cpu-is-at-100-now-what</a></li>
<li><a href="https://scoutapm.com/blog/which-rails-profiler-is-right-for-you">https://scoutapm.com/blog/which-rails-profiler-is-right-for-you</a></li>
<li><a href="https://github.com/MiniProfiler/rack-mini-profiler">https://github.com/MiniProfiler/rack-mini-profiler</a></li>
<li><a href="http://maratgaliev.com/profiling-ruby-rails/">http://maratgaliev.com/profiling-ruby-rails/</a></li>
<li><a href="https://engineering.appfolio.com/appfolio-engineering/2019/6/4/where-does-rails-spend-its-time">https://engineering.appfolio.com/appfolio-engineering/2019/6/4/where-does-rails-spend-its-time</a></li>
</ul>Claas ZurawskiWhile investigating performance problems in a Rails app, a profiler is quite helpful. stackprof is such a tool which can be added to an existing Rails app quite easily.Straw Man Alerting System: logcheck2018-09-07T17:25:00+02:002018-09-07T17:25:00+02:00https://www.claasloader.de/devops/2018/09/07/straw-man-alerting-system-logcheck<p>Monitoring application log files is important. If something goes wrong, you want to be notified. This is even more critical for background jobs where no user is facing the error directly.</p>
<p>A state of the art solution for this problem would be <a href="https://www.elastic.co/what-is/elk-stack">ELK</a>, combined with <a href="https://www.elastic.co/what-is/elasticsearch-alerting">Alerting</a> , but maybe your infrastructure is not at this point yet. For small systems or as a workaround solution, there’s an old Linux tool called <code class="highlighter-rouge">logcheck</code>.</p>
<p>Logcheck was actually developed to monitor Linux system log files (e.g., detect a hacker attack), but it can be easily configured for your own application logs as well.</p>
<p>What logcheck can do:</p>
<ul>
<li>every x hours, it would scan a list of log files you specify</li>
<li>each line in the logfile is matched against a list of regular expressions you specify</li>
<li>all the lines which are matched would be sent out by e-mail</li>
</ul>
<h2 id="installation">Installation</h2>
<p>On Ubuntu, install it like</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">~<span class="nv">$ </span><span class="nb">sudo </span>aptitude <span class="nb">install </span>logcheck</code></pre></figure>
<p>Logcheck gets configured in <code class="highlighter-rouge">/etc/logcheck</code>. This directory looks like</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>.
├── cracking.d
├── cracking.ignore.d
├── ignore.d.paranoid
├── ignore.d.server
├── ignore.d.workstation
├── violations.d
├── violations.ignore.d
├── header.txt
├── logcheck.conf
└── logcheck.logfiles
</code></pre></div></div>
<p>For our setup, only</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>.
├── ignore.d.server
├── violations.d
├── logcheck.conf
└── logcheck.logfiles
</code></pre></div></div>
<p>are needed.</p>
<h2 id="setup">Setup</h2>
<p>Edit <code class="highlighter-rouge">logcheck.conf</code> and enter the e-mail address for your alerts.
We keep <code class="highlighter-rouge">REPORTLEVEL="server"</code> unchanged, so logcheck will run in <code class="highlighter-rouge">server</code> mode.
Therefore, we can ignore directories <code class="highlighter-rouge">ignore.d.paranoid</code> and <code class="highlighter-rouge">ignore.d.workstation</code> completely.</p>
<p>Add all the logfiles you want to monitor to <code class="highlighter-rouge">logcheck.logfiles</code>. Placeholders work:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># /etc/logcheck/logcheck.logfiles
/var/log/my_app/*.log
</code></pre></div></div>
<p>Now, add a new file <code class="highlighter-rouge">violations.d/my_app</code> and enter your regex, e.g.:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># /etc/logcheck/violations.d/my_app
^.*error.*$
^.*Error.*$
^.*exception.*$
^.*Exception.*$
</code></pre></div></div>
<p>Finally, ignore everything else. Add</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code># /etc/logcheck/ignore.d.server/my_app
^.*$
</code></pre></div></div>
<p>to <code class="highlighter-rouge">ignore.d.server/my_app</code>.</p>
<h2 id="disclaimer">Disclaimer</h2>
<p>Be aware of these caveats:</p>
<ul>
<li>it doesn’t scale: it can only act on logs from a single server</li>
<li>it can only do regex, cannot detect patterns on multiple rows</li>
<li>for low volume alerts only</li>
<li>no UI</li>
<li>no history (except the e-mail themselves)</li>
</ul>
<p>However, until you have something like ELK, it might serve you well.</p>Claas ZurawskiMonitoring application log files is important. If something goes wrong, you want to be notified. This is even more critical for background jobs where no user is facing the error directly.Dynamic Error Pages in Rails: You’re still doing it wrong2018-02-23T18:00:00+01:002018-02-23T18:00:00+01:00https://www.claasloader.de/programming/ruby/rails/2018/02/23/error-pages-in-rails-youre-still-doing-it-wrong<p>Error pages are important. Well, so important, there are even <a href="http://a.co/12NSyBn">books</a> written about it.</p>
<p>For your shiny new Rails app, you most often want to customize the error pages (404 and 500). Rails’ <a href="https://github.com/rails/rails/blob/ef0b05e78fb0b928c7ef48d3c365dc849af50305/railties/lib/rails/generators/rails/app/templates/public/500.html">standard pages</a> are not appropriate for most use cases (how could they).</p>
<p>Usually, you want to include your page logo image, some common page headers, and at least one link back into your application. The layout of the error page should be at least similar to the rest of the application. Now there’s a problem: By default, error pages are static html files. This makes totally sense, since in case of serious server problems, your Rails app might not be able to create the error page dynamically anymore. On the other hand, you cannot reference your standard CSS resources in a static html file since you don’t know the Asset Pipeline ids in advance.</p>
<p>When you google this problem, people recommend pretty complicated strategies like:</p>
<ul>
<li><a href="https://medium.com/ruby-on-rails-web-application-development/custom-400-500-error-pages-in-ruby-on-rails-exception-handler-3a04975e4677">define your own exception app</a></li>
<li><a href="https://mattbrictson.com/dynamic-rails-error-pages">monkey patch Rails</a></li>
<li><a href="https://medium.com/@tair/custom-error-pages-in-rails-you-re-doing-it-wrong-ba1d20ec31c0">use a gem to create pages without Asset Pipeline ids</a></li>
<li><a href="https://pooreffort.com/blog/custom-rails-error-pages/">just use ERB and hope that your Rails app would always be alive to serve the error page</a></li>
</ul>
<p>Instead, I recommend a much simpler solution:</p>
<ul>
<li>Add controller actions for 404 and 500</li>
<li>Use ERB or Haml to style your error pages nicely</li>
<li>Right after deployment, call your 404 and 500 actions and save them as static HTML files</li>
</ul>
<p>It would require only a few lines in Capistrano (thanks to my colleague <a href="https://github.com/ArslanAle">Ali</a> for figuring out the details):</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="c1"># ./lib/capistrano/tasks/error_pages.rake </span>
<span class="n">namespace</span> <span class="ss">:error_pages</span> <span class="k">do</span>
<span class="n">desc</span> <span class="s1">'Generate static error pages and save in public folder'</span>
<span class="n">task</span> <span class="ss">:create</span> <span class="k">do</span>
<span class="n">on</span> <span class="n">roles</span><span class="p">(</span><span class="ss">:web</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">host</span><span class="o">|</span>
<span class="n">public_500_html</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="n">release_path</span><span class="p">,</span> <span class="s2">"public/500.html"</span><span class="p">)</span>
<span class="n">execute</span> <span class="ss">:curl</span><span class="p">,</span> <span class="s1">'-s'</span><span class="p">,</span> <span class="s2">"https://www.example.com/500_page"</span><span class="p">,</span> <span class="s2">"> </span><span class="si">#{</span><span class="n">public_500_html</span><span class="si">}</span><span class="s2">"</span>
<span class="n">public_404_html</span> <span class="o">=</span> <span class="no">File</span><span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="n">release_path</span><span class="p">,</span> <span class="s2">"public/404.html"</span><span class="p">)</span>
<span class="n">execute</span> <span class="ss">:curl</span><span class="p">,</span> <span class="s1">'-s'</span><span class="p">,</span> <span class="s2">"https://www.example.com/404_page"</span><span class="p">,</span> <span class="s2">"> </span><span class="si">#{</span><span class="n">public_404_html</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">after</span> <span class="s1">'deploy:published'</span><span class="p">,</span> <span class="s1">'error_pages:create'</span></code></pre></figure>Claas ZurawskiError pages are important. Well, so important, there are even books written about it.Integrate a Time Attendance System2015-05-22T15:27:00+02:002015-05-22T15:27:00+02:00https://www.claasloader.de/sysadmin/programming/architecture/2015/05/22/integrate-a-time-attendance-system<p>Some weeks ago we got the requirement to integrate our punch in/out door device with our main business system. When our people come to work, they would hold their ID cards in front of the machine, and the machine would create records who arrived to work and at what time. Now the task is to read out data from the door machine and display it in the main system.</p>
<p>Sounds easy? Well, in theory, it is.</p>
<p>In practice, the first problem is to choose an appropriate time attendance model. Given the requirement that we want to read out the data by our own software, the ideal model would be one which has some kind of web service (HTTP) API. We could then regularly call this API to import the door punches into our system.</p>
<p>Unfortunately, we were not able to find any single product supporting a HTTP API. All of the products we checked come with their own kind of time management software, typically some standalone application running on Windows, where you can create your users, setup working shifts, holidays etc. This software can connect to the device and calculate the working times.</p>
<p>When it comes down to APIs, about half of the products we’ve checked would be kicked out directly. They just didn’t offer any. The only way to read data out would be to used the software shipped with the device. Those which do offer an API would all ship with a SDK for C#/.NET.</p>
<p>Although we are a Ruby and not a .NET shop, we decided for a Fingertec Timeline 100. They have a fairly comprehensive website, offer a lot of different software to use with their devices, and claim their SDK being “well documented”. When you check their support sites and their documentation, you’ll find that quality is not that high. However, compared with the other manufacturers we examined, they looked best.</p>
<p>Checking Fingertec’s SDK and API documentation, it looked like using it would be days of pain. Fortunately, they offer some software which we could use instead of the SDK. The FTDP (Fingertec Data Processor) can be installed as a Windows service, reading the punch data regularly and writing it to a database (SQL Server or Oracle). Our software could then connect to that database, and we’re done.</p>
<p>So we setup FTDP and SQL Server as described in their guide. Sadly, the Windows service always failed to connect to the database. We’ve checked this problem with Fingertec’s support (they were really nice and responsive) and they confirmed that it’s a bug in the current FTDP version (1.7).</p>
<p>Finally, we came up with a hacky solution: Running FTDP as a Windows service is totally fine when using the local Access database (which is the default setup). You can then access the data by converting the Access .mdb file into CSV. We copy the <code class="highlighter-rouge">FTDP.mdb</code> file to our Linux server and use <a href="http://mdbtools.sourceforge.net/install/x53.htm">mdb-tools</a> for conversion. mdb-tools provides the mdb-export command, so you could dump the data like</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>mdb-export /path/to/file.mdb Transaction <span class="nt">-D</span> <span class="s2">"%Y-%m-%d %H:%M:%S"</span></code></pre></figure>
<p>To copy the file, we setup the FTDP data directory as a network share and mount it on the Linux server via Samba. (Update: In the meantime, we use AWS S3 to transfer the data.)</p>Claas ZurawskiSome weeks ago we got the requirement to integrate our punch in/out door device with our main business system. When our people come to work, they would hold their ID cards in front of the machine, and the machine would create records who arrived to work and at what time. Now the task is to read out data from the door machine and display it in the main system.Install Stash 2 on Ubuntu 12.042014-05-23T17:11:00+02:002014-05-23T17:11:00+02:00https://www.claasloader.de/stash/ubuntu/linux/sysadmin/2014/05/23/install-stash-2-on-ubuntu-12-dot-04<p>If you need an inhouse git server and you have a small team only, then <a href="https://www.atlassian.com/software/stash">Stash</a> from Atlassian might be something for you. We use it for several months now and I can say we are pretty happy with it.</p>
<p>Here’s how to install it on a Ubuntu 12.04 box:</p>
<h3 id="prerequisites">Prerequisites</h3>
<p>Stash is running on Java, so you need to install a JRE. OpenJDK is supported.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">~<span class="nv">$ </span><span class="nb">sudo </span>aptitude <span class="nb">install </span>openjdk-7-jre-headless</code></pre></figure>
<p>Git is also needed, of course:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">~<span class="nv">$ </span><span class="nb">sudo </span>aptitude <span class="nb">install </span>git</code></pre></figure>
<p>You also need MySQL in case you want to run the database on the same server:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/srv<span class="nv">$ </span><span class="nb">sudo </span>aptitude <span class="nb">install </span>mysql-server-5.5</code></pre></figure>
<h3 id="prepare-user-and-directories">Prepare User and Directories</h3>
<p>Stash needs its own user, whose home directory is used as a data store. I therefore chosed to locate this directory under <code class="highlighter-rouge">/srv/</code>:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/srv<span class="nv">$ </span><span class="nb">sudo mkdir </span>stash
/srv<span class="nv">$ </span><span class="nb">sudo</span> /usr/sbin/useradd <span class="nt">--create-home</span> <span class="nt">--home-dir</span> /srv/stash <span class="nt">--shell</span> /bin/bash stash
/srv<span class="nv">$ </span><span class="nb">sudo chown </span>stash:stash stash/</code></pre></figure>
<h3 id="install-files">Install Files</h3>
<p>I chosed to host installation in <code class="highlighter-rouge">/opt/</code>:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/opt<span class="nv">$ </span><span class="nb">sudo cp</span> <span class="nt">-a</span> /home/claas/install/atlassian-stash-2.11.3
/opt<span class="nv">$ </span><span class="nb">sudo chown</span> <span class="nt">-R</span> stash:stash atlassian-stash-2.11.3/</code></pre></figure>
<h3 id="set-stash_home">Set STASH_HOME</h3>
<p>You need to tell Stash where its home directory is located. Add following line to <code class="highlighter-rouge">/opt/atlassian-stash-2.11.3/bin/setenv.sh</code></p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">set </span><span class="nv">STASH_HOME</span><span class="o">=</span>/srv/stash</code></pre></figure>
<h3 id="ipv4">IPv4</h3>
<p>Stash now already starts, but I noticed it listens on IPv6 ports only. To enable the regular IPv4 ports, open ``/opt/atlassian-stash-2.11.3/bin/setenv.sh` again and add the following parameter to the appropriate Java call:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nt">-Djava</span>.net.preferIPv4Stack<span class="o">=</span><span class="nb">true</span></code></pre></figure>
<h3 id="try-stash-the-first-time">Try Stash the First Time</h3>
<p>Now, Stash is ready for a first try. Start it by</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/opt/atlassian-stash-2.11.3/bin<span class="nv">$ </span><span class="nb">sudo</span> <span class="nt">-u</span> stash <span class="nt">-i</span> /opt/atlassian-stash-2.11.3/bin/start-stash.sh</code></pre></figure>
<p>And open <code class="highlighter-rouge">http://<your-server>:7990/setup</code> to see if it’s working.</p>
<p>You can stop it by</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/opt/atlassian-stash-2.11.3/bin<span class="nv">$ </span><span class="nb">sudo</span> <span class="nt">-u</span> stash <span class="nt">-i</span> /opt/atlassian-stash-2.11.3/bin/stop-stash.sh</code></pre></figure>
<h3 id="automatic-startup">Automatic Startup</h3>
<p>We want to have Stash being started automatically once the server is rebooted.</p>
<p>First, create a generic name for the application directory (this will upgrading to a newer version easier later on):</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/opt<span class="nv">$ </span><span class="nb">sudo ln</span> <span class="nt">-s</span> atlassian-stash-2.11.3 atlassian-stash-latest</code></pre></figure>
<p>Download the <a href="https://bitbucket.org/ssaasen/atlassian-stash-lsb-startup-script/src/master/stash">startup script provided by Atlassian</a> and save it as <code class="highlighter-rouge">/etc/init.d/stash</code>. Adjust following lines to match our installation:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">...
<span class="c"># Required-Start: $remote_fs $syslog $mysql</span>
...
<span class="nv">STASH_INSTALLDIR</span><span class="o">=</span><span class="s2">"/opt/atlassian-stash-latest"</span>
...
<span class="nv">STASH_HOME</span><span class="o">=</span><span class="s2">"/srv/stash"</span></code></pre></figure>
<p>And enable it via:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/etc/init.d<span class="nv">$ </span><span class="nb">sudo </span>update-rc.d stash defaults
/etc/init.d<span class="nv">$ </span><span class="nb">sudo chmod</span> +x /etc/init.d/stash</code></pre></figure>
<p>Finally, we need to set the <code class="highlighter-rouge">JAVA_HOME</code> environment variable. Add this line to ` /etc/environment`:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">JAVA_HOME</span><span class="o">=</span><span class="s2">"/usr/lib/jvm/java-7-openjdk-amd64"</span></code></pre></figure>
<h3 id="install-jdbc-driver">Install JDBC Driver</h3>
<p>Stash doesn’t come with a JDBC driver, so you have to download this separately from the <a href="http://dev.mysql.com/downloads/connector/j/">MySQL home page</a>. Then, copy it to the stash installation directory:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/opt/atlassian-stash-latest/lib<span class="nv">$ </span><span class="nb">sudo cp</span> /path/to/mysql-connector-java-5.1.29/mysql-connector-java-5.1.29-bin.jar <span class="nb">.</span>
/opt/atlassian-stash-latest/lib<span class="nv">$ </span><span class="nb">sudo chown </span>stash:stash mysql-connector-java-5.1.29-bin.jar </code></pre></figure>
<h3 id="prepare-database">Prepare Database</h3>
<p>Finally, you need to create a database for Stash:</p>
<figure class="highlight"><pre><code class="language-sql" data-lang="sql"><span class="n">mysql</span><span class="o">></span> <span class="k">SET</span> <span class="k">GLOBAL</span> <span class="n">storage_engine</span> <span class="o">=</span> <span class="s1">'InnoDB'</span><span class="p">;</span>
<span class="n">Query</span> <span class="n">OK</span><span class="p">,</span> <span class="mi">0</span> <span class="k">rows</span> <span class="n">affected</span> <span class="p">(</span><span class="mi">0</span><span class="p">.</span><span class="mi">00</span> <span class="n">sec</span><span class="p">)</span>
<span class="n">mysql</span><span class="o">></span> <span class="k">CREATE</span> <span class="k">DATABASE</span> <span class="n">stash</span> <span class="n">CHARACTER</span> <span class="k">SET</span> <span class="n">utf8</span> <span class="k">COLLATE</span> <span class="n">utf8_bin</span><span class="p">;</span>
<span class="n">Query</span> <span class="n">OK</span><span class="p">,</span> <span class="mi">1</span> <span class="k">row</span> <span class="n">affected</span> <span class="p">(</span><span class="mi">0</span><span class="p">.</span><span class="mi">02</span> <span class="n">sec</span><span class="p">)</span>
<span class="n">mysql</span><span class="o">></span> <span class="k">GRANT</span> <span class="k">ALL</span> <span class="k">PRIVILEGES</span> <span class="k">ON</span> <span class="n">stash</span><span class="p">.</span><span class="o">*</span> <span class="k">TO</span> <span class="s1">'stash'</span><span class="o">@</span><span class="s1">'localhost'</span> <span class="n">IDENTIFIED</span> <span class="k">BY</span> <span class="s1">'secret'</span><span class="p">;</span>
<span class="n">Query</span> <span class="n">OK</span><span class="p">,</span> <span class="mi">0</span> <span class="k">rows</span> <span class="n">affected</span> <span class="p">(</span><span class="mi">0</span><span class="p">.</span><span class="mi">00</span> <span class="n">sec</span><span class="p">)</span>
<span class="n">mysql</span><span class="o">></span> <span class="n">FLUSH</span> <span class="k">PRIVILEGES</span><span class="p">;</span>
<span class="n">Query</span> <span class="n">OK</span><span class="p">,</span> <span class="mi">0</span> <span class="k">rows</span> <span class="n">affected</span> <span class="p">(</span><span class="mi">0</span><span class="p">.</span><span class="mi">00</span> <span class="n">sec</span><span class="p">)</span>
<span class="n">mysql</span><span class="o">></span> <span class="n">quit</span></code></pre></figure>
<h3 id="end">End</h3>
<p>These steps were enough to setup a basic Stash server for our needs. Check the <a href="https://confluence.atlassian.com/display/STASH/Installing+Stash+on+Linux+and+Mac">manual</a> for more installation options and details.</p>Claas ZurawskiIf you need an inhouse git server and you have a small team only, then Stash from Atlassian might be something for you. We use it for several months now and I can say we are pretty happy with it.Emacs Macro for Current Region2014-05-23T17:09:00+02:002014-05-23T17:09:00+02:00https://www.claasloader.de/emacs/2014/05/23/emacs-macro-for-current-region<p>Recording a macro in Emacs is a well known tasks, but today I struggled with applying the macro for the current region only. Thanks to <a href="http://stackoverflow.com/a/15166731/384689">this answer</a> on <a href="http://stackoverflow.com">SO</a>, I ended with the following recipe:</p>
<ol>
<li>Select region</li>
<li><code class="highlighter-rouge">F3</code></li>
<li><code class="highlighter-rouge">M-x narrow-to-region</code></li>
<li>… operations on region, <code class="highlighter-rouge">M-<</code> to jump to beginning of region, <code class="highlighter-rouge">M-></code> to the end</li>
<li><code class="highlighter-rouge">M-x widen</code></li>
<li><code class="highlighter-rouge">F4</code></li>
</ol>
<p>When you want to save your macro for later usage, run</p>
<ol>
<li><code class="highlighter-rouge">M-x name-last-kbd-macro Enter <mymacro> Enter</code></li>
<li>Open <code class="highlighter-rouge">.emacs</code></li>
<li><code class="highlighter-rouge">M-x insert-kbd-macro Enter <mymacro> Enter</code></li>
<li>Add line <code class="highlighter-rouge">(global-set-key [f9] '<mymacro>)</code> to bind <code class="highlighter-rouge">F9</code> to your macro</li>
</ol>Claas ZurawskiRecording a macro in Emacs is a well known tasks, but today I struggled with applying the macro for the current region only. Thanks to this answer on SO, I ended with the following recipe: