Incremental regeneration in latest jekyll build
jekyll is an awesome static generator but its regenetion time is totally unexceptable. It works well when you have small number of posts but start to degrade as number of posts increases. Actually, I don’t care about it much when I
git push to the repo because my VPS is using SSD-cached but previewing the site at
localhost is such a pain.
So when I take a look at
jekyll again today and see this pull request, I’m so excited and couldn’t help but clone the repo and rebuild
jekyll to test it out myself.
git clone https://github.com/jekyll/jekyll.git gem build jekyll.gemspec gem install jekyll*.gem
A quick look at
jekyll reveals that it now has a file named
.jekyll-metadata to keep track of
mtime of the change file and their dependencies.
D:/Source/myawesomeblog.com/_posts/2015-01-31-welcome-to-jekyll.markdown: mtime: 2015-01-31 12:31:57.000000000 +07:00 deps: - /Source/myawesomeblog.com/_layouts/post.html - D:/Source/myawesomeblog.com/_includes/head.html - D:/Source/myawesomeblog.com/_includes/header.html - D:/Source/myawesomeblog.com/_includes/footer.html - /Source/myawesomeblog.com/_layouts/default.html
I tested the beta gem on an old PC of mine with 5400rpm HDD to see how much speedup the new build offers. Prior this, when using
jekyll serve, the regeneration speed is really slow. It takes minutes (literally minuteS) for a site with barely over 100 posts.
It does actually speedup. A quick change in a single file now takes 17 to 40 seconds to regenerate (of total ~100 posts). Still no where near the speed I expect but it’s a big step up from where it was. I’ll take it :)
This pull request is expected to be in
jekyll 2.6.0 release.
BitTorrent Sync as an alternate backup solution
I’ve heard so many good feedbacks about BitTorrent Sync. The only complain seems to be how it isn’t open-source. Today, I’ve decided give BitTorrent Sync (btsync) a whirl to see what all the hypes are about.
The way it works is you first select a folder you want to share. btsync will create two keys: one for readonly access and the other one for read/write access. You then pass on the key to the person you want to share the file with. They paste it to their btsync client and that’s all, btsync will do the rest.
Here I thought why don’t I use it to backup stuff to my VPS. My Dropbox is just a bit over 70GB and the 50GB free space promo is going to expire soon. I’m going to need to offload some stuff elsewhere, namely Camera Uploads. BitTorrent Sync seems to the perfect fit for this kind of thing. Plus, I have quite a lot of free space on my VPS.
So I installed btsync on my VPS. It’s as simple as download the file and extract. You will get an executable
# generate a sample config ./btsync --dump-sample-config >> btsync.conf ./btsync --config btsync.conf
btsync by default will start on port
8888. I then create an Apache2’s virtualhost, bind it to port 8888 in order to access its webui.
From my local machine, I add a folder to btsync, copy its readonly access key and add it to my server’s btsync webui. Tick
Override any changed files on the server.
BitTorrent Sync doesn’t support document revision yet so Dropbox is still far superior. But the sync speed it offers is just amazing, leaving Dropbox in the dust.
Though I don’t trust my server keeping my data safe, it doesn’t hurt to have one more alternative backup. You definitely shouldn’t use it as your sole backup solution.
Theming Apache's mod_autoindex directory listing
By default, your shared folder with
mod_autoindex enabled looks like this.
The icons looks like they were from Windows 3.1 era. I am by no mean a designer but looking at this seriously make me want to vomit.
I decided to take a quick look at
mod_autoindex’s documentation and grab some free icons from Dribbble to make it better.
Turns out, it’s easier than I thought. Theming is supported out-of-the-box with
mod_autoindex. I just have to copy some icons (frequently used file types) and add a few lines in my Apache’s vhost config. I can even inject CSS too. Every element already has proper id and class so I can just inspect the page and customize to my liking.
IndexIgnore /_theme # hide _theme folder AddIcon /_theme/icons/blank.png ^^BLANKICON^^ AddIcon /_theme/icons/folder.png ^^DIRECTORY^^ AddIcon /_theme/icons/upper_level_directory_icon.png .. AddIcon /_theme/icons/image.png .png .jpeg .gif .jpg DefaultIcon /_theme/icons/fallback_icon.png IndexStyleSheet "/_theme/style.css" # injecting CSS HeaderName /_theme/header.html # customize header ReadmeName /_theme/footer.html # footer
Here’s how mine looks like after.
If you’re too lazy for this stuff and just want a decent looking theme, take a look at Apaxy. Follow the installation instruction there.
Some of the most useful tips I learn when working with NodeJS
modules management with npm
Nodejs comes with an amazing package manager called
npm. Start a project with
npm init which will then create a configuration file named
package.json, keeping track of all the modules your project is using. You don’t have to manually manage this file yourself. If you want to add a package to
package.json you can add
--save parameter when installing it.
npm install koa-static --save
Also, you should ignore
npm_modules folder when
git push because whoever clone the repo can do
npm install by themselves.
npm_modules folder can grow pretty big so no one would want to
git clone the whole thing.
pm2 instead of forever
When you start learning about NodeJS, you may notice that node process may exit unexpectedly when errors are not handling properly. forever is a node package that ensure node process will run continuously in the background. But forever is very limited. It doesn’t have support for clustering, very limited logging and monitoring.
I later found out a much leaner solution called
pm2. Compare to
pm2 looks like a full solution for deployment with builtin clustering support, terminal configuration and better logging support. In fact, ever since, I only use
pm2 for my production server.
Enabling clustering with
pm2 is as easy as
pm2 start app.js -i 0
0 means that
pm2 will utilize number of threads equals to number of your CPU’s cores. You can specify the no. of child process as you want, ideally one per processor core.
As for local development, I prefer nodemon to keep track of changes in my application and automatically restart the server.
For starters, callbacks are nightmare. Many popular frameworks still make use of callback heavily which creating the sense for newbies that it is the correct way of doing things in
nodejs. It’s not. Over the last 2 months, I’ve started using
Q, async, bluebird and then generators. Of those, generator seems to be the most elegant solution, producing much more readable code than callbacks.
Debugging with node-inspector
I’m pretty sure most nodejs starters will use
console.log() everywhere to debug the application. I did that. It was ok for quick debugging but sometimes, you need a little more than just
console.log. node-inspector is a node package based on Blink Developer Tools that let you debug right in your favorite browser.
node-inspector and use
node-debug app.js for debugging.
npm install node-inspector -g node-debug app.js
These are some of the things that I’ve learnt in the last 2 months working with nodejs. Nodejs is an amazing platform but it can be a pain sometimes. These tips save me a lot of times and make developing in nodejs so much more bearable.
Install io.js on Mac OS X
iojs is not going to be a flop. Look at the current development rate, I would say
iojs would replace
nodejs eventually if Joyent not doing anything to turn it around.
I prefer to install
nvm instead of the default installer package on iojs’s homepage since it allows me to switch between node version with a single command.
As of current,
iojs is basically a dropin replacement for
nodejs so you don’t have to worry much.
nvm - a node version manager with
curl https://raw.githubusercontent.com/creationix/nvm/v0.23.2/install.sh | bash
nvm install iojs
Add these lines to your
export NVM_DIR=~/.nvm source $(brew --prefix nvm)/nvm.sh%
iosjs as the default node version
nvm alias default iojs
You can check again with
which node to see which version of node is using.
Things software developers wish they had known in their 20s
The era in which a common career trajectory is to become a lifer at some software company and work one’s way up to higher and higher internal positions has been over for a long time (This era had already been over in the 1990s when I was a young programmer).
You don’t owe the company you work for anything beyond what you put your name on when you signed the employee agreement.
Don’t stick around too long if the place you are working for isn’t doing much for your bottom line, is not your dream job, and isn’t adding anything new to your resume; that is, the easiest way to get a promotion and a substantial raise is to switch jobs. Don’t switch jobs if you like what you are doing but also don’t stay somewhere only out of loyalty to a company.
If you think 2. and 3. are cynical and if looking at things this way makes you feel like an asshole, console yourself with the knowledge that the company you work for views you in exactly the same way and will act on such a view in a second if it is in its interest to do so.
You can usually skip all hands meetings even if they are supposedly mandatory.
Keep an eye on corporate politics. Being a successful computer programmer is about social engineering as much as it is about software engineering.
Make sure you are at the meetings at which the scope and/or requirements are defined for any piece of software for which you are going to be responsible.
Don’t go dark. Check in code often – with stubs or whatever if necessary. It is better to check in code sooner rather than later despite the fact that doing so might entail dealing with transitory problems. A few hiccups now are better than a gigantic merge conflict down the line.
Be careful about your estimates on dates and on how long things are going to take. Understand that the whole game is about dates. When in doubt just triple what you actually think even if it secretly sounds absurd to you.
Don’t be religious about languages, libraries, or platforms.
Except for Perl. Perl sucks.
At a certain point it is you the programmer who will decide when things are done, even if it isn’t officially supposed to be your decision. Ultimately there is a point before releases when you have to push back at PM or whoever and say, “What you are asking for is not a bug fix – it is a feature request. We are way beyond the point at which we can even be considering introducing new features” It will always be you the programmer who does this because no one else will.
Don’t check in a lot of code right before a vacation or long weekend.
It is easier to not get assigned to something that you don’t want to work on than it is to get out of it after it has been assigned to you. Trust your instincts regarding death marches. If some project doesn’t feel right, don’t get assigned to it.
If you find yourself working for a place at which there is a manager who is printing out burn-down charts and hanging them on the wall, start looking for another job.
Working for big companies/name software companies can suck more than you can possibly imagine. Just because your friend who works for Company X is always bragging about the rock climbing wall doesn’t mean that Company X is a great place to work.
For all but the simplest cases, prefer recursive descent parsers over regular expressions.
Get good at debugging and learn to use a profiler.
Focus on making your way into a position in which you are doing what you like software-wise while you are young and can more easily take risks.
Don’t break the build. If you do break the build, don’t make a lot of excuses – no one cares – just fix it quickly.
In order to get link without Quora censoring stuff, just add any
?abc= in the url. My personal favorite is
?fuck=you, in the hope that it will be popular enough to show in their log someday :D
Bill had been a Wall Street quant and had “Vice President” in his title, noting that VP is a mid-level and often not managerial position in an investment bank. His current title was Staff Software Engineer, which was roughly Director-equivalent. He’d taught a couple of courses and mentored a few interns, but he’d never been an official manager. So he came to me for advice on how to appear more “managerial” for the VP-level application.
A fascinating read by Michael O. Church. Lots of advices on software development careers.
Great blog. Subscribed!
It’s interesting how much movies can change our perception: survey asking people in France who contributed the most to the defeat of Nazi Germany.
Fix WELD-001408 unsatisfied dependencies for type error when deploying to GlassFish
Took me half an hour today to figure this out when deploying to production servers. Apparently, the CDI extension loaded from a JAR in a WAR, thus different classloader, makes it uninjectable.
The problem can be fixed by simply disable implicit CDI on GlassFish
StrongLoop allows you to quickly create REST APIs using their graphic interface and CLI. SLC also supports debugging, profiling, tracing, deploying as well as monitoring features.
Creating REST APIs with
slc is as easy as creating datasource and model. StrongLoop will do the rest for you.
I was very tempted to use StrongLoop for a recent project but I had to deal with a legacy database that use a very old
odbc driver which force me to use