alberto | Rant

Destructive git behaviour

I destroyed all the work I had done in a project for the last 2 months

tl;dr:

GIT doesnt consider the files in .gitignore important and will happily replace them

Im pretty careless with my local git commands

Ive been trained by git to be this careless. Unless i use --force on a command, git will always alert me if im about to do something destructive. Even then, worse case scenario, you can use git reflog to get back in time after a bad merge or something not easily accesible with a normal git flow

What happened?

I had a link to a folder in my master branch. I branched to do some work and decided to replace the link with the actual folder to untangle some other mess and added it to .gitignore to avoid git complaining about it

Then happily worked on in for 2 months

I was ready to merge it, so I made a final commit and I checked out master

So far, pretty normal git flow… right?

But wait, something was wrong. My folder was missing!

Wait, what?! what happened!

The folder existed as a syslink on master, so git happily replaced my folder with a now broken syslink

It seems git doesnt consider files under .gitignore as important

You can see by yourself and reproduce this behaviour by typing the following commands. It doesnt matter if links doesnt exists:

[~/tmp]
$ mkdir gitdestroy/

[~/tmp]
$ cd gitdestroy/

[~/tmp/gitdestroy]
$ cat > file1
hi, im file1

[~/tmp/gitdestroy]
$ ln -s nofile link

[~/tmp/gitdestroy]
$ ll
total 48K
drwxr-xr-x. 26 alberto alberto  36K Jan 29 15:18 ..
-rw-r--r--   1 alberto alberto   13 Jan 29 15:19 file1
lrwxrwxrwx   1 alberto alberto    6 Jan 29 15:19 link -> nofile
drwxr-xr-x   2 alberto alberto 4.0K Jan 29 15:19 .

[~/tmp/gitdestroy]
$ git init
Initialized empty Git repository in /home/alberto/tmp/gitdestroy/.git/

[~/tmp/gitdestroy (master #%)]
$ git add -A

[~/tmp/gitdestroy (master +)]
$ git status
On branch master

No commits yet

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)

    new file:   file1
    new file:   link


[~/tmp/gitdestroy (master +)]
$ git commit -m "link on repo"
[master (root-commit) 5001c61] link on repo
 2 files changed, 2 insertions(+)
 create mode 100644 file1
 create mode 120000 link

[~/tmp/gitdestroy (master)]
$ git checkout -b branchwithoutlink
Switched to a new branch 'branchwithoutlink'

[~/tmp/gitdestroy (branchwithoutlink)]
$ git rm link 
rm 'link'

[~/tmp/gitdestroy (branchwithoutlink +)]
$ mkdir link

[~/tmp/gitdestroy (branchwithoutlink +)]
$ cat >link/file2
hi im file2

[~/tmp/gitdestroy (branchwithoutlink +%)]
$ cat > .gitignore
link

[~/tmp/gitdestroy (branchwithoutlink +%)]
$ git status
On branch branchwithoutlink
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

    deleted:    link

Untracked files:
  (use "git add <file>..." to include in what will be committed)

    .gitignore


[~/tmp/gitdestroy (branchwithoutlink +%)]
$ git add -A

[~/tmp/gitdestroy (branchwithoutlink +)]
$ git commit -m "replace link with folder"

[branchwithoutlink 2cfb06c] replace link with folder
 2 files changed, 1 insertion(+), 1 deletion(-)
 create mode 100644 .gitignore
 delete mode 120000 link

[~/tmp/gitdestroy (branchwithoutlink)]
$ ll
total 60K
drwxr-xr-x. 26 alberto alberto  36K Jan 29 15:18 ..
-rw-r--r--   1 alberto alberto   13 Jan 29 15:19 file1
drwxr-xr-x   2 alberto alberto 4.0K Jan 29 15:21 link
drwxr-xr-x   4 alberto alberto 4.0K Jan 29 15:22 .
-rw-r--r--   1 alberto alberto    5 Jan 29 15:22 .gitignore
drwxr-xr-x   8 alberto alberto 4.0K Jan 29 15:22 .git

[~/tmp/gitdestroy (branchwithoutlink)]
$ git checkout master
Switched to branch 'master'                                        <--- NO ERROR???

[~/tmp/gitdestroy (master)]
$ ll
total 52K
drwxr-xr-x. 26 alberto alberto  36K Jan 29 15:18 ..
-rw-r--r--   1 alberto alberto   13 Jan 29 15:19 file1
lrwxrwxrwx   1 alberto alberto    6 Jan 29 15:22 link -> nofile    <--- WHAT
drwxr-xr-x   8 alberto alberto 4.0K Jan 29 15:22 .git
drwxr-xr-x   3 alberto alberto 4.0K Jan 29 15:22 .

[~/tmp/gitdestroy (master)]
$ git checkout branchwithoutlink 
Switched to branch 'branchwithoutlink'

[~/tmp/gitdestroy (branchwithoutlink)]
$ ll
total 56K
drwxr-xr-x. 26 alberto alberto  36K Jan 29 15:18 ..
-rw-r--r--   1 alberto alberto   13 Jan 29 15:19 file1
-rw-r--r--   1 alberto alberto    5 Jan 29 15:23 .gitignore
drwxr-xr-x   8 alberto alberto 4.0K Jan 29 15:23 .git
drwxr-xr-x   3 alberto alberto 4.0K Jan 29 15:23 .

Aftermath

I analyzed what git was doing underneath in hopes to gain some insight on how to recover these files. It seems git unlinkat(2) everyfile and finally rmdir(2) the folder

By contrasts rm(1) just uses unlinkat(2) in every file and folder

Not sure what difference this makes, but it was quite useless. I tried some EXT undelete tools to try to recover the missing files, but everything was gone

Actually I was able to undeleted some files i had removed 3 years ago that i didnt need :/

Future

This directory was under git as well and remotely hosted. But my last push was 2 months ago. I will be more careful on the future

Recently theres been some discussion on git about something that could prevent this behaviour. They are introducing the concept of “precious ignored” files

But for me the damage was done

This was unexpected behaviour for me. Maybe it was also for you. Be safe out there!

No more bash

I recently stopped my (imho bad) habit of starting shell scripts in bash; no matter how small the task at hand feels originally

I had an epiphany

The number of bash scripts that grew out of control was just too damn high

ive been told that

it's a difficult balance

But is it really?

Its always the same story

Well, I only have to run the same handful of commands multiple times in different directories, a shell script will do
Except, sometimes it fails when…/this special case if../oh, never considered this… I will just add a couple more lines and fix it
…
Script explodes, and gets rewritten in python

It rarely had exceptions for me. Almost every .sh (if its intended to automate something) had to do sanity checks, error control/recovery and probably special case scenarios… eventually

Im aware that if you are well versed in bash, you can do a lot. It has arrays and all kind of (imho weird) string mangling to make advanced use of variables, but it always felt like bash was filled with pitfalls that you have to learn to route around

Writting the same thing in python takes about the same time. Maybe a couple more lines to write a few imports

Im aware that python comes with its own pitfalls, but at least you can actually scale it when needed. And you save the rewritting part

This is really hard for me to say

I grew to love long one liners of pipes that solve complex problems. Also, most of the time you only seem to want to run a couple of commands on tandem

But I think its time for me to say goodbye. In the same way I said goodbye to perl (and thats a rant for another day :))

No more shell scripts

No matter how small

rant

Latest posts related to :