Inside every folder that you’ve ever opened in the Finder* is a hidden file, .DS_Store, poised there waiting to trip you up. It contains private data for the Finder, and has been quietly causing havoc since the first release of Mac OS X over 22 years ago.
DS_Store stands for Desktop Services Store, a file in which macOS keeps information about folder settings, icon locations, Finder (or Spotlight) Comments for files within that folder, and more. It was introduced when the code for the Finder was rewritten under the technical leadership of Arno Gourdol in 1999. The intention at the time was that these Desktop Services would be given a public API, but that never happened, and what’s stored in all those .DS_Store files remains opaque. Attempts to reverse-engineer its structure haven’t proved straightforward as it turns out to be quite complex, with different records stored in a B-tree. In any case, the importance of .DS_Store files isn’t so much in what they contain, as how they can trip up apparently simple things.
Over the years, .DS_Store files have caused problems with backups being made to NAS and other network storage systems, with revision control systems, folder archives, websites, and some local backups. They’ve tripped me up too, as they can readily be copied across via AirDrop, then replace an existing .DS_Store file with an older one, which in turn confuses some backup utilities. There are even reports that some users have had .DS_Store files flagged for breaching copyright when they have been stored in Google Drive.
When you create and manipulate directories in Terminal and scripts alone, without going near the Finder, you shouldn’t find .DS_Store files being created or updated. The moment that you open them in the Finder, or possibly over a network using SMB or AFP, each folder will have a .DS_Store file created within it. You can’t see those in the Finder, even if you enable hidden files to be shown, because they have also been given an ‘invisible’ flag since macOS Sierra, but they’re listed in Terminal and by some third-party file browsers.
Sometimes they’re confused with another annoying but more useful type of hidden file: shadow files whose names start with ._ and are used to carry extended attribute data as part of the AppleDouble file format used on some FAT file systems. They too are invisible in the Finder even when hidden files are supposed to be displayed, but are associated with individual files rather than folders.
While there isn’t much you can do about them on your Mac, given that they’ll only return the moment the Finder notices they’re missing, there are specific workarounds that have been devised to prevent them from spreading to other systems. If you use Git, for instance, Mikey @0xmachos has explained how you can suppress them effectively. In scripts it’s often worth testing file names for .DS_Store, and when it’s found, ignoring their nuisance. Forcing the Finder to refresh them is easily accomplished: open the folder in the Finder and reshuffle the layout of icons, for instance.
Recent versions of macOS should no longer write .DS_Store files to computers connected to them over a network. If you want to stop them from being exposed in network volumes of older systems, use the command
defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool true
to disable that. You can recursively delete .DS_Store files from a hierarchy using the command
find . -name .DS_Store -delete
or in the app BlueHarvest, which supports macOS Ventura.
Perhaps their greatest frustration is that they’re where Finder (or Spotlight) Comments are stored. As I explained a few days ago, this duplicates what is also shown in a file’s com.apple.metadata:kMDItemFinderComment extended attribute, but takes priority over it, and isn’t well synchronised. It’s not difficult to end up with a file with conflicting information in the local .DS_Store file and its FinderComment extended attribute. When that does occur, using Spotlight to search for a string in the Finder/Spotlight Comment can fail, or behave unpredictably.
So in summary they’re invisible hidden files which you’re not supposed to know about. You can’t do much about them, so when they do trip something up, be prepared to massage that folder until its .DS_Store behaves again.
Further reading:
Wikipedia on .DS_Store files
* Thanks to Jeff Johnson for pointing out that you need write access to that folder, and that some change must have been made to that folder in the Finder, such as creating or copying a file into that folder, for the .DS_Store file to be created by the Finder.