Earlier today, I posted about the ability to find previously deleted emails using the Spotlight search on the iPhone, adding to the already intense swarm of news surrounding this issue. My findings were that, by ensuring your trash folder was emptied and refreshed, the problem appeared to be solved.

However, many of you responded indicating that this did not work for you, and that you were still seeing cached messages. Then Cult of Mac posted an update to their original piece, suggesting that the problem was more specific to POP accounts. Although I had tested both POP and IMAP accounts with similar results, I decided to investigate further.

First, to clarify: The problem here is *not* with Spotlight caching the contents of your email, as many have suggested. Spotlight simply indexes the available content on your phone, and logs a pointer to it. When you tap a search result, it opens the associated application and tells that app to view the content. In this case, it sees an email message and asks Mail.app to open it, which it does, because Mail.app still has the message content stored locally. Secondly, this appears to be specific to POP accounts. On IMAP or Exchange accounts, deleted email messages do appear in Spotlight results until the trash folder is emptied, as described in my earlier post.

I created a fresh account and configured it for POP access, then downloaded the mail. Searching spotlight, I could see the message I was looking for. I then went back to the message and deleted it, and that's where the fun starts. I conducted this same test multiple times, and found that sometimes, the message would now show up in Spotlight twice, one result opened the message, intact, with the Inbox listed as the folder to return to; the other displaying a message in the trash folder, but with an error displaying the body of the message. Other times, I would only see one result, pointed directly to the trash.

Note: The problem discussed below behaves differently depending on the type of email account being used. Users of POP mail accounts are indeed noticing a problem with deleted email messages appearing in Spotlight search results. Please see this post for updated information.

Our friends over at Engadget have highlighted a Cult of Mac post that, at first glance, appears to have found a bug in Spotlight's caching of email search results on the iPhone. The idea is that after you delete an email from your inbox in the iPhone's Mail.app client, you can still locate and open the e-mail using Spotlight search, if you know the subject of the message.

What's really happening, though, is what Apple might otherwise call "user confusion." When you delete an e-mail message in most mail clients, the message isn't magically deleted, but instead moved to a "trash" or "deleted messages" folder. Being a folder, it is indexed by Mail to provide an unread count, as well as by Spotlight so that you can easily find messages in the folder. When you search for the email, it shows up in Spotlight and opens in Mail because the message still exists in your trash folder.

All right, so what can you do to avoid this? Well, you need to empty the trash folder from your email. With many providers, you can just go to the trash folder, and delete the message from there to permanently remove it. The problem there is that, if you use a service like Gmail, the message doesn't really get deleted when you do this because of the way Gmail translates its labels into folders. Gmail's Trash folder is also set to purge itself after 30 days by default.

Your options? You either have to use the Gmail web interface in Safari, or if you don't mind tweaking your Gmail settings, you can enable the Advanced IMAP Controls Lab and then change your Gmail settings to immediately delete messages instead of using the trash folder. (Note that if you have your "All Mail" label configured to show up in IMAP, you will need to either disable it, or delete the message from that folder on the iPhone as well. See this thread for more information.) Once you've deleted the message, go in to the Trash folder and hit the refresh button to ensure that the message is no longer listed, which will remove it from the local mail cache. This should also clear up Spotlight's index so that the message no longer appears.

By now, you're probably wondering if this is really something to worry about. In my opinion, it really depends on how you manage your email. If you rarely delete messages, or you just delete things that are not of interest to you, they will most likely disappear from your trash folder after a specified time period that is set by your email provider (usually somewhere between 7 and 30 days.) However, if you intentionally delete messages with the idea that the message is unretreivable, then you should probably consider testing this out for yourself to see if it affects you. Again, with many providers, just periodically deleting the contents of the trash folder will do the trick. If you use Gmail, or your company's email services are hosted by Google, then you may need to log in to the web interface periodically to empty the trash, or you can look into the available configuration options, if you feel comfortable doing so.

So in reality, this isn't really a bug as much as it is what I would call "intended, but unexpected behavior." Many people simply forget that the trash folder exists, particularly on the iPhone where you don't always see it sitting off to the left hand side of your inbox. Checking your trash folder for any unintentionally deleted items, and cleaning out the old messages periodically, isn't really a bad habit to get into. With that said, however, I do think that Apple could improve this behavior by implementing some simple changes like adding an "Empty Trash" button to the trash folder, showing the count of all messages in the trash instead of just unread messages (so that it's easier to notice there are messages in there), and having Spotlight either ignore deleted messages like Leopard's Spotlight does, or at least indicate that they were found in the trash.

What are your thoughts on this? Do you consider this to be a bug, or do you think it's just a place that Apple could improve the experience on? Let us know in the comments!

Update: Leander Kahney has posted an update to the original Cult of Mac post that clarifies this issue a bit more. It seems that accounts using POP are configured by default to remove messages after one day or one week, thus causing them to continue to appear in Spotlight search results. Unfortunately, there isn't an option to remove them immediately, however, only after a day, one week, one month, or never. Because of this, Spotlight still sees them in the index, and Mail still contains a cached copy of the message until the time limit configured in the settings is reached.

If you'd like to check out this setting for yourself, go to the Settings app, tap on Mail, Contacts & Calendars, and choose the POP mail account you want to modify. Scroll down and choose Advanced, and the retention settings will be at the top of that page under Deleted Messages. It appears the default for a new account is to retain messages up to one week. So this is still the OS doing what it's configured to do, but there really should be an option to remove messages immediately here. If Apple were to add an option there, and/or remove Spotlight's ability to index deleted messages, the problem would be solved.

Update 2: After further investigation, it appears that some messages deleted on POP accounts do disappear from Spotlight, while others show up long after the message is deleted. I have conducted several tests on this situation, including on different versions of the iPhone OS, and have posted an update with details.

We've discussed some pretty neat uses for Automator workflows in the past, but did you know you can easily integrate Spotlight searches into your automation workflows? Well, you can, and it might be easier than you think. In this Mac Automation post, I'm going to show you how you can integrate it into a workflow that will burn the resulting files to a CD or DVD.

Creating the workflow
To create this simple workflow, you'll need to use two actions. Find the "Spotlight" action and drag it to your workflow section; then, find the "Burn a Disc" action and drag it below the newly added Spotlight action. That's it; your workflow is now completed, but we need to tweak the actions.

Using the workflow
To test the workflow, you can enter a search term in the Spotlight action, and a Disc name in the Burn a Disc action. Insert a rewritable disc, or a regular CD/DVD, and click the "Run" button in the upper right corner of the Automator window. Automator will then go to work finding all of the search terms that matched files, then will then burn all found items to the disc. This could take a while to burn, depending on the resulting files.

Remember that all of the Spotlight tricks, such as exclusion, denoting kind, etc. are still available even though you're using the Automator action.

If you've been listening to all the clamoring on Twitter, or perhaps have seen the updated Apple page, then you know there was an iPhone event today to discuss a new version of the iPhone OS, version 3.0. While you will have to wait until the summer to get the new version (unless you join the developer program), Apple showed off over 100 new features that we're all excited about. iPhone developers are also getting some love from Apple with over 1,000 new APIs.

Some of the biggest updates include:

• Push notifications will be standard in iPhone 3.0. Apple claims that there was a scaling issue when they first announced this feature last year, with thousands of developers clamoring to use it, so it had to be delayed. Developers will be able to build message, sound, and badge alerts into their applications.
• iPhone will support cut, copy and paste -- copy text and pictures, then paste them in any application on the device. If you change your mind about the newly pasted data, just shake to display a menu asking if you wish to undo the paste.
  
• In-App purchasing -- developers will have the ability to charge a fee from within the application for more levels in a game, magazine subscriptions, eBooks, etc. This solves the problem of the 20 ebook readers that only allow you to read one book each.
  
• MMS -- you'll be able to use MMS on your iPhone 3G (yes, 1st gen iPhones will not get the fortune of using MMS due to the use of a different cell radio). The new Messages application will be taking the place of the SMS app and will allow you to send voice memos and pictures to your friends.
  
• Peer-to-Peer connectivity -- allows iPhone apps the ability to find other iPhone users over WiFi or Bluetooth. This can be using for anything from gaming to sharing data.
  
• Third-party Accessories -- app developers can now access third-party accessories attached to the iPhone. This could allow for developers to interface with their own accessories to create a truly unique experience on the device. One of the demonstrated uses was a prototype blood glucose meter for the iPhone, from a Johnson & Johnson subsidiary, which is already getting attention from our readers; it's not for sale yet!
  
• Apple will now be allowing applications the ability to get GPS coordinates from Core Location and turn that into software that provides Turn-by-Turn directions for iPhone users. However, due to the terms of use with Google maps, developers will be required to license their own maps for this use (meaning that Turn-by-Turn GPS applications will almost certainly be paid apps). This isn't a problem for vendors like TomTom or Telenav, who already have licenses for their map data.
• A landscape keyboard option is now available in all iPhone applications, including the infamous Mail.app.
• Voice memos will allow iPhone users the ability to capture audio and then email it, or send it through the Messages application as a multimedia message.
• System-wide Spotlight searching is built right into the home screen. To the left of the first home screen, there is a magnifying glass, so by flicking to the left of the first home screen will show this search app. You can search for anything, including mail, music, and notes. There is no word yet on how this will interface with third-party applications.
  
• Stereo Bluetooth audio devices are now supported through the use of A2DP technology.

iPhone owners should note that A2DP technology and the use of MMS will not be available on 1st generation iPhones. This is probably due to the iPhone Bluetooth spec used, and Apple has confirmed (during the Q&A session after the keynote) that MMS is not available because of the cell radio used.

iPhone OS 3.0 will be available for regular users "this summer." iPhone developers will be able to get the beta of OS 3.0 sometime today -- Apple is still busy updating their iPhone Dev Center, but we'll let you know when it is available.

After waiting a day for the dust to settle -- and for developers to catch up -- since Apple's release of Safari 4 beta, it looks like there are relatively few show-stopping bugs to talk about. Just about every problem we've heard about at TUAW, and every issue I've had personally, has been an issue with a plugin. Fortunately, for those of us who really depend on hacks plugins for surfing productivity, most developers have been preparing for the release using nightly builds and have already come through with updates.

Yes, plugins break when software is updated, when the software is beta, and when -- due to the lack of a genuine plugin architecture -- plugins are, in fact, hacks. The developers of our third-party goodies have our backs, in most cases, and I expect the ripples on the water to die down completely over the next week.

In the area of search plugins, I switched from Inquisitor to Glims a while ago because of the extra features Glims offers. I haven't seen an update to Inquisitor yet, but Glims came out with build 13 this morning. Unfortunately, my search-as-I-type functionality still isn't working, but the plugin loads and the additional functionality is there. It's quite possibly a conflict with another plugin ... not that I'm running that many.

Gravity Applications has released a native iPhone application to complement Searchlight, their remote search application based on Spotlight. Searchlight previously sported an iPhone-optimized web interface, and the native app builds on that. Some noteworthy benefits of the native app include storing logins for multiple servers, Bonjour discovery, and a speedier interface than Safari had provided. It also integrates a new feature found on the Searchlight server: the ability to send the files you locate to yourself or a third party via email.

The iPhone app is basic, but it provides a solid foundation for building a richer feature set as it develops, according to developer Niclas Bahn. We discussed some cool features already existing in some of the iPhone file management apps like FileMagnet and Briefcase (iTunes links), and he was excited about implementing some innovations of his own. It will be fun to watch it develop.

The Searchlight iPhone application is free, but requires Searchlight to be running on the computer you'd like search access to. Current users will want to upgrade their Searchlight server to the 2.1 release (the upgrade is free). If you weren't one of the lucky ten readers who got a free copy of Searchlight last time we mentioned it, and the $99 price tag was a bit too high, there's some good news: the price has dropped to $49USD. It may be a temporary price-drop, but it should last long enough for new users to take the demo for a spin and make an informed decision. Get it at Gravity's website, and happy searching!

Are you tired of accidentally opening spotlight by pressing command + space? With this simple Terminal "hack," you can rid your precious menu bar of Spotlight for once and all. Just open Terminal.app (located in /Applications/Utilities/) and type the following command:

sudo chmod 0 /System/Library/CoreServices/Spotlight.app

You will be required to authenticate as an administrator, then the command will run. While you are still in the Terminal, type "killall Spotlight" to turn off the spotlight service right then. Spotlight will suddenly dissappear from your menu bar. To get Spotlight back, just type the following command back into Terminal:

sudo chmod 775 /System/Library/CoreServices/Spotlight.app

After a few seconds, Spotlight will reappear in your menu bar. Note that this does not remove Spotlight from Finder windows, only from the menu bar.


Want more tips and tricks like this? Visit TUAW's Mac 101 and Terminal Tips sections.

Searchlight 2.0 was officially released last week. Searchlight offers remote Spotlight functionality, allowing you to search your computer's files via a web browser, and it includes an iPhone-optimized interface. I talked with Searchlight's authors, Gravity Applications, during WWDC and got a good look at the pre-release version which left me pretty impressed. I've personally been using the beta for a while now to access the Mac Mini at my house which serves as my central file repository. Read on for a Searchlight overview ... and an exclusive TUAW giveaway!

More Mac 101, our series of tips and tricks for new and returning Mac users. Love it or hate it, Spotlight's ubiquitous search has changed the Mac OS X experience; quick and (mostly) reliable full-text indexing is now part of the landscape. It's not just for finding files, however; did you know that Spotlight makes a handy application launcher too?

You can launch most any application by hitting the Spotlight hotkey (?-Space by default), then typing the first few letters of the application's name. Chances are it will be the "top hit" for your search, so go ahead and hit Return or Enter to launch it immediately. This approach doesn't have the power-user moxie of third-party launchers like QuickSilver, Butler or LaunchBar, but it's helpful for applications that don't live in your Dock when you want to reach them quickly.

If you've got ideas, suggestions or questions on the Mac 101 front, just let us know below.

Welcome to another entry in our series for new Mac users: Mac 101. This installment brings us to the preference panel of Spotlight, OS X's integrated search that makes finding that needle in your hard drive's haystack as easy as pie (clearly, it doesn't protect against mixed metaphors).

You're familiar with the way Spotlight returns results, but did you know that you can change the order in which those results are shown? Read on to find out how (and why you might want to do this).

Just the other day I noticed an odd thing: the Spotlight magnifying glass on my MacBook Pro had that little pulsating dot in it that means it is indexing something. That was odd, since I hadn't added any large files to my Mac, nor had I connected an external harddrive. Or did I?

As you might be able to guess from the picture above, Spotlight was dutifully trying to index my Time Capsule's harddrive every time that drive was mounted by Time Machine (which is once an hour). This process was making my backups take a little longer than I wanted them to, so I simply excluded my Time Capsule from Spotlight's indexing.

Here's how you do it:

1. Launch System Preferences (or you can open the Spotlight preferences from the Spotlight dropdown menu).
2. Click on the 'Privacy' tab.
3. click the little plus icon ()
4. Choose your Time Machine disk (as you can see mine is cleverly named)

That's it! Now Spotlight will ignore your Time Machine disk, your backups will run a little faster, and there will be a spring in your step.

Update: As some commenters have noted, this tip does mean that you won't be able to use Spotlight to search your Time Machine backups. That's fine for me, since when I'm using Time Machine I'm restoring a file that I noticed is missing, or an entire folder. I don't need to search for something, however, if you find yourself routinely searching your Time Machine backups via Spotlight this tip isn't for you.

Google's Mac group has released Precipitate, a plugin which allows you to search Google Docs and Google Bookmarks from the same interface that you search you Mac. It works with both Spotlight and Google Desktop for Mac. Basically the developer's idea is to have a one stop search interface for information on your Mac and information in the Google cloud so you don't have to remember where you saved something. Sounds like a good idea to me.

Google's Precipitate is a free download from Google Code.

Here's a very simple and even more useful tip from Dennis Best. He's thought of a clever way to tag almost any document he creates in Leopard with Spotlight and a clever use of keywords.

When writing a note, composing an email message, etc., he precedes keywords with the ° bullet character (shift-option 8). Later, he can conduct a Spotlight search for " °Tahoe" and find every tagged email message, sticky note, iCal event, etc. Pretty clever!

For added usefulness, save that Spotlight search for a self-updating reference.

[Via Micropersuasion]

Some applications remind me of cars. For example, most Java based applications are Volkswagen Things as they're useful in certain situations, but horrifying (or hilarious) to look at. But once in a while you get a Jaguar XJS: a beautiful exterior that doesn't bely its underlying power. That, and you frequently have to spend some time at the mechanic. Leap by Ironic Software – makers of Yep – just hit 1.0 after a long stint in beta. It's my current Jag.

We covered the beta, but here's a quick review. Leap is a Spotlight interface which, once you get used to it, causes a fundamental shift in the current paradigm of file management. Prior to finding Leap, I had already been using a homegrown, Spotlight-based file management system. The strategy – based on a tagging system – requires a far smaller folder hierarchy and allows for much faster location of related files in a project. Leap both complements my system and elegantly replaces a lot of its kludgier functionality.

By combining full Spotlight search capabilities with a tagging system (that writes to Spotlight comments), Leap makes file location a breeze and allows for easy association of files within a project. It can function like Finder, showing all files in a single directory, if that's what you need at the moment. The "Go Deep" search can display every file in every sub-folder of a project. The results can be filtered by criteria such as file type, location, date, keywords and other metadata, allowing for quick searching and tagging. It can also display a traditional list format, but the icon view with Loupe function and Quick Look integration makes browsing as elegant as any program I've tried. And, of course, a tag cloud view – and the ability to choose tags from a sidebar to narrow the search – makes all that tagging work worthwhile.

All of that being said, Leap still has some bugs despite its 1.0 status. I've been subjected to freezes fairly frequently, but the random crashing that plagued the beta seems to be cured. As far as features go, I wish there were an easy way to exclude parameters from a query. I'm also wishing for the ability to select multiple file types during a search using a ?-click rather than defining custom file type groups. But the interface, as it stands, is friendly and elegant once you get the basic concepts down. And nitpicking aside, it does elegantly accomplish everything I've been doing in a more manual fashion.

Leap is now selling for $59. You can buy Leap and Yep as a bundle for $69, and previous owners of Yep can get Leap for $39. That works for people who got Yep in a bundle as well. You can download a trial copy from Ironic Software and take it for a spin.

Erica mentioned mdfind a little while ago. It's not a brand new tool, but there hasn't been a great deal of discussion surrounding it. For shell scripters and Terminal junkies, it's a goldmine of search goodness that can make you never want to key out find . -name "letsgogetcoffeewhilewewait.rb" or even locate didcronupdatethedb.yet again.

There are two mdfind options mentioned at Mac OS X Hints that are worth a mention, one found only in the man page, and one found only in the usage info. The former being "-interpret", which allows the advanced Spotlight query syntax to be used on the command line as if typed into the Spotlight window, turning mdfind "kMDItemContentType == 'public.jpeg'" into mdfind -interpret "kind:jpeg". Its opposing force would be "-literal", which forces a literal interpretation of the query string. Any qualified Spotlight query should be usable with "-interpret", so go ahead with mdfind -interpret "kind:image flash:1 modified:<=1/1/08 ex-girlfriend" to find those late-night pictures of last year's special someone. That would take a lot of metadata work to pull off without "-interpret".

The second option mentioned isn't in the man page, but you'll see it in the usage info if you run mdfind with no parameters. It's the "-s" option that loads saved searches. It will look in ~/Library/Saved Searches unless a path is specified. So you can save a search in Spotlight called "PDFs from yesterday" and run mdfind -s "PDFs from yesterday" to launch the same search at the command line. See, I had to throw a "normal" search in there to keep you on your toes.

Terminal jockeys can mix these up with a few pipes, stir in some perl and sprinkle in a little awk to get some really great Spotlight soufflé. Of course, if you don't like the command line and you're still reading this, you're a downright hardy trooper and I commend you. Go have a cup of coffee, you deserve it.

[via Mac OS X Hints]