Saturday, September 17, 2011

Bejeweled autopilot

The idea behind the Bejeweled game is as following:
An average game status looks like this:

The player is allowed to switch between any two neighboring gems in order to create a series of three or more gems of the same color. If the player does so, the gems are destroyed, points are given, gems are falling down to fill the empty spaces, and new gems are coming from the top to fill the vacuum. The problem is that the player has just about one minute to destroy as many gems as possible, and it’s hard to find series of gems that quick.
What I did to solve this problem was to write a small Python script to capture the screen, find the best next move and simulate mouse clicks to execute it. One of the problems I faced while writing the script was identifying the kinds of gems, while the bustards kept on moving, rotating n’ glowing in different kinds of effects that usually inform the player about special bonuses. I tried to scan a 4x4 pixels square in the middle of any square, to get a color value out of this small square, and to compare it against a table I made manually in order to decide which gem it is.
I found out, that in many cases bots scripts alter the game to change all the graphics to something more OCR friendly. But I didn’t want to do that, as I’m trying to perfect the technique of writing a bot, and not to mix the two game cheating vectors. Therefore, I didn’t want to put any patches on the game, just writing a bot.
The next problem I faced was Python itself, as I found out that it’s quite slow to scan an image or to compare two images with the Python Image Library (PIL). Though, it is possible that I’m miss-using the library. Please, I would like to hear any ideas on how this part could be done better from Python, without extending Python with a custom C library.
One lessen I learnt the hard way during writing of this very script, was that it’s very important to find the time to write a code for finding the location of the game on the screen, so in case I scroll the page down a little, or just open the bookmarks tools bar, it won’t be using the hard coded position of the game and miss figure where the gems are.
Even thought, I successfully set myself on the top score among all of my friends, but not world wide best just yet. I found out that many fine tunings on the scripts, such as what strategy to prefer, weather to aim for bonuses first, try to destroy a longest chain of gems at once, or save bonuses for later, have immense effects on scoring. Please, I would love to hear from the readers of this blog about any ideas on how to improve the script, or how you would write one in case you had to.

The script is aviable on my SVN at



  1. Anonymous25/9/11

    There was a reddit discussion about bejeweled pwnage (, they also had the same problem as your with correctly identifying the images, I do not recall they found an optimal solution for all the situations, but it was quite likely because none of them have researched enough.
    However, in order to improve score the main idea behind was to find more combos with intelligent guessing (this truly reminds me of tetris attack) faster

    Have fun

    have fun.

  2. Anonymous18/10/11

    Yay! another entry :)

  3. I saw the Reddit post, they done some amazing things there. The only thing that I don't understand is that they refer to 1 million points as something that is really hard to achieve, but my dumb script that uses nothing fancy or shiny, has manged to get the 1 million points many times.