The .data dir and the Editor

by Andrew Chilton


Posted a year ago in category platform.


Whilst writing a short FAQ for Glitch (yet to be released, but watch this space) I started writing an entry for the following question.

Why do files created in .data/ disappear from the editor?

My answer in the FAQ is going to be the following:

In short they don't disappear completely, they only disappear from the editor view. This happens after your app sleeps.

However this isn't the whole story since I ended up going down a rabbit hole to figure out more of how the .data dir and the editor interact. Here's what I found.

In general the editor shows the files in the editor that Git knows about on the server. If you'd like to see this then open your console (in 'Advanced Settings') and run the following command (this is the current state of this ga-faq project):

$ git ls-files
.config/configstore/update-notifier-npm.json
.glitch-assets
License.md
ReadMe.md
lib/app.js
lib/faqs.js
package.json
public/s/css/styles.css
public/s/js/app.js
server.js
views/index.pug

However the .data dir is ignored by Git due to configuration set up by Glitch. We can see this by doing the following on the server:

app@ga-faq:~ 21:01 
$ touch .data/new.txt
app@ga-faq:~ 21:03 
$ git add .data/new.txt 
The following paths are ignored by one of your .gitignore files:
.data/new.txt
Use -f if you really want to add them.

Righto, we'll try that.

app@ga-faq:~ 21:04 
$ git add -f .data/new.txt 
app@ga-faq:~ 21:06 
$ git ls-files | grep new
.data/new.txt

That seems to work, but the new.txt file doesn't yet show up in your editor. Since this isn't automatic yet, you can force the editor to refresh the app list by running the following:

$ refresh

$ git ls-files | grep new
.data/new.txt

Bah! Even though we've added that file to git, and git still knows about it, the file doesn't appear in the editor. This seems to be something which Glitch has strong opinions on, which is fine, since the .data/ dir isn't copied when you remix a project so this seems like the safest option.

Let's tidy-up by telling git to unstage this new file:

$ git reset HEAD .data/new.txt

Now we've tried this by adding a file to git on the server, so let's try it in the editor too. Go and click + New File and type in .data/another.txt. Bam, we can see it in the editor.

In this case, neither git ls-files nor git status knows about the file, however it is showing up in the editor. This is a good indicator of the answer to our original question.

Since you added the file in the editor then it knows to show you it in that current session. However, the next time your app sleeps and your editor either loads or reloads then the file will disappear from the editor view, because git doesn't know about it.

But don't fret, since that file still exists on the server, which you can see with the following command:

$ ls -l .data/another.txt
-rw-r--r-- 1 app app 0 Dec 11 21:12 .data/another.txt

The upshot is, don't expect any files in .data/ to be copied upon remix (which we already knew) but also don't expect any files in there to be versioned with git either, and hence be shown in the editor in any meaningful way. However, do expect that any files in there will be persisted within your project space on the server under /app/.data as is expected.

One final thing I'd like to investigate is where the rule telling Git to ignore .data lives. Usually there is a .gitignore file at the top-level of the git repo:

app@ga-faq:~ 21:17 
$ pwd
/app
app@ga-faq:~ 21:17 
$ ls -l .gitignore
ls: cannot access '.gitignore': No such file or directory

Perhaps there is something in the .git/config which gives us an indication of where this rule might live:

app@ga-faq:~ 21:18 
$ cat .git/config 
[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[user]
        email = none
        name = Glitch (ga-faq)

Nothing there either, so let's check the GIT_* environment variables. We can type echo $GIT and then press tab a couple of times for autocomplete:

app@ga-faq:~ 21:19 
$ echo $GIT<tab><tab>

After being stumped for a while, Tim Kington helped in pointing out that there is a global .gitignore file used in your Glitch project. Let's have a look:

$ git config --list | grep excludes
core.excludesfile=/etc/gitignore-global

Ace. We've got it. And it wouldn't be us if we didn't take a peek inside:

$ cat /etc/gitignore-global 
*.log
*.pid
*.pid.lock
*.pyc
*.seed
*.tgz
.bash_history
.bundle
.cache
.cargo
.data
.env
.eslintcache
.glitch-meta
.gomix-meta
.grunt
.local
.lock-wscript
.next
.node-gyp
.node_repl_history
.npm
.npmrc
.nuget
.nyc_output
.ssh
.subversion
.yarn-integrity
bower_components
build/Release
coverage
jspm_packages/
lib-cov
logs
node_modules/
npm-debug.log*
package-lock.json
pids
target
typings/
yarn-debug.log*
yarn-error.log*

And finally there we have it. The .data directory is there just as we expected. It was certainly the long way round to find this but I think we've had a good adventure doing it.


Tags: editor, data.