Folder Structure & File Naming after reading up on the French way of Archiving PART 1

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Folder Structure & File Naming after reading up on the French way of Archiving PART 1

meikamona
I went away and read up on the French way of Archiving. I've decided to follow it. 
It was pretty close to how I was doing it lower down in the food chain.  On reading it, the first thing it allowed me to do was to ignore, for the moment, the difference between the folder structure (the pathway) and the filename. One can put any and all of the information in either or both places. This let me clarify a lot of crosstalk about repositories, sources and authorities, as well as between authors and subjects.


I use this top level model description of the pathway & filename, it has three broad parts made up of several elements each.
MetaSource_DATE_DataPoints

Each of these group elements which can be used in the pathway for folders, or filenames, and/or both.

MetaSource

is comprised of the elements:

     ①Repository (where is the book)

    ②Source (what book)

    ③Authority. (Who wrote the book)

MetaSource attempts to describe where in the world the data arises. Crosstalk can arise because each real world thing can be each, two-of or all-three of these things.


DATE

gives the MetaSource a time.

    2017-02-17

    can be YEAR

    or YEAR-MONTH

    or YEAR-MONTH-DAY.

Also “about”, “ca”, “after” or “before” can also be added. (Timestamps would have to use “-” instead of “:” because Microsoft Windoze).

Pad with ‘0’ see https://en.wikipedia.org/wiki/ISO_8601

The DATE is the link &  balance between the big picture of where in the world described in MetaSource and what the DataPoint/s to.


DataPoints

is an arrangment of:

    ①Subject:

        ⓐperson/s

        ⓑevent:

        birth

    death

    marriage

    etc

        ⓒplace

        ⓓtitle

    ②Type:

    map

    photo

    scan

    downloaded

    screenshot

can include secondary -DATE

    ③MISC:

    original file name

    original URI


Filenaming can include all of the groups’ elements, or some, the rest being put in the folder names of the pathway. Folder Structure can include some of the groups’ elements or none. The choice depends on the complexity of the resources’ origins and file subjects.


Generic Pathways & File Namespaces

A template for a pathway and file name is as follows:

REPOSITORY-GID_SOURCE-GID_AUTHORITY-GID_DATE_SUBJECT-GID_TYPES_ORIGINAL FILE NAME or URI

GRAMPS IDs (GIDs)s are optional.

 Implementation includes, with pathway in bold, file name plain:

~/media&files/REPOSITORY/SOURCE_AUTHORITY-GID_DATE_LastNameFirstname_portrait_photo.jpg

~/media&files/REPOSITORY/SOURCE/AUTHORITY-GID_DATE_LastNameFirstname_portrait_photo.jpg

~/media&files/REPOSITORY/SOURCE/AUTHORITY-GID/DATE_LastNameFirstname_portrait_photo.jpg

~/media&files/REPOSITORY/SOURCE/AUTHORITY-GID/DATE/LastName Firstname_ portrait_ photo .jpg

The above are all equivalent from the point of view of MetaSource_DATE_DataPoints.


Implemented Examples:

For a downloaded image from a digital archive:

~/media&files/TROVE_KerryCH_ca1890_MrMontaguesGreenHillsStation-photo_download_nla.obj-141608194-1.jpg

Notice here TROVE is both Repository and source. If an element has multiple roles then in a file name the element need not be repeated.

As TROVE is likely to be used a lot, it will be given it’s own folder. Repositories with single files will not be given folders unless this leads to very long file names.

~/media&files/TROVE/TROVE_KerryCH_ca1890_MrMontaguesGreenHillsStation-photo_download_nla.obj-141608194-1.jpg


It is possible for all elements of the MetaSource to be the same entity.

A good example is The New South Wales Registry of Births Deaths & Marriages (or NSW-BDM-R0001 below) because it is the repository, the source and the authority all rolled into one.

~/media&files/NSW-BDM-R0001 /NSW-BDM-R0001-1909-02-14-WhitnallJim-I003-Birth-extract-scan.jpg

~/media&files/NSW-BDM-R0001/1909-02-14-WhitnallJim-I003-Birth-extract-scan.jpg     -------(I'll probably use this one)

Remember elements of this structure can be used for either folder or filenames or both, to indicated relationships clearly, and depending on requirements, to reduce pathway and filename length.

And the last example is an image of a church in an ebook at  http://gutenberg.net.au/ebooks06/0600151h.html

~/media&files/GutenbergNetAuR0015/Gutenberg.Net.Au-R0015_SydneyIn1948_FowlesJoseph_1948_StPhilipsChurchHill_engraving-Plate 10B_download-2017-02_0600151h_html.jpg

You can find the real books here http://trove.nla.gov.au/work/9403137 


 ---I don't think I'll put in the page plate reference in my pathway/filename at this stage. It about 160 CHAR long, I think Windoze machines can handle a pathway& filename total length of about 260 CHAR.


Anyway---------

This is the general schema. I call my familiar archives fonds now in GRAMPS as repositories.


I'll put pesky detail in the next email.


meika

http://meika.loofs-samorzewski.com


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Folder Structure & File Naming after reading up on the French way of Archiving PART 1

Peter Merchant
On 28/02/17 06:18, meika loofs samorzewski wrote:
I went away and read up on the French way of Archiving. I've decided to follow it. 
It was pretty close to how I was doing it lower down in the food chain.  On reading it, the first thing it allowed me to do was to ignore, for the moment, the difference between the folder structure (the pathway) and the filename. One can put any and all of the information in either or both places. This let me clarify a lot of crosstalk about repositories, sources and authorities, as well as between authors and subjects.


I use this top level model description of the pathway & filename, it has three broad parts made up of several elements each.
MetaSource_DATE_DataPoints

Each of these group elements which can be used in the pathway for folders, or filenames, and/or both.

MetaSource

is comprised of the elements:

     ①Repository (where is the book)

    ②Source (what book)

    ③Authority. (Who wrote the book)

MetaSource attempts to describe where in the world the data arises. Crosstalk can arise because each real world thing can be each, two-of or all-three of these things.


DATE

gives the MetaSource a time.

    2017-02-17

    can be YEAR

    or YEAR-MONTH

    or YEAR-MONTH-DAY.

Also “about”, “ca”, “after” or “before” can also be added. (Timestamps would have to use “-” instead of “:” because Microsoft Windoze).

Pad with ‘0’ see https://en.wikipedia.org/wiki/ISO_8601

The DATE is the link &  balance between the big picture of where in the world described in MetaSource and what the DataPoint/s to.


DataPoints

is an arrangment of:

    ①Subject:

        ⓐperson/s

        ⓑevent:

        birth

    death

    marriage

    etc

        ⓒplace

        ⓓtitle

    ②Type:

    map

    photo

    scan

    downloaded

    screenshot

can include secondary -DATE

    ③MISC:

    original file name

    original URI


Filenaming can include all of the groups’ elements, or some, the rest being put in the folder names of the pathway. Folder Structure can include some of the groups’ elements or none. The choice depends on the complexity of the resources’ origins and file subjects.


Generic Pathways & File Namespaces

A template for a pathway and file name is as follows:

REPOSITORY-GID_SOURCE-GID_AUTHORITY-GID_DATE_SUBJECT-GID_TYPES_ORIGINAL FILE NAME or URI

GRAMPS IDs (GIDs)s are optional.

 Implementation includes, with pathway in bold, file name plain:

~/media&files/REPOSITORY/SOURCE_AUTHORITY-GID_DATE_LastNameFirstname_portrait_photo.jpg

~/media&files/REPOSITORY/SOURCE/AUTHORITY-GID_DATE_LastNameFirstname_portrait_photo.jpg

~/media&files/REPOSITORY/SOURCE/AUTHORITY-GID/DATE_LastNameFirstname_portrait_photo.jpg

~/media&files/REPOSITORY/SOURCE/AUTHORITY-GID/DATE/LastName Firstname_ portrait_ photo .jpg

The above are all equivalent from the point of view of MetaSource_DATE_DataPoints.


Implemented Examples:

For a downloaded image from a digital archive:

~/media&files/TROVE_KerryCH_ca1890_MrMontaguesGreenHillsStation-photo_download_nla.obj-141608194-1.jpg

Notice here TROVE is both Repository and source. If an element has multiple roles then in a file name the element need not be repeated.

As TROVE is likely to be used a lot, it will be given it’s own folder. Repositories with single files will not be given folders unless this leads to very long file names.

~/media&files/TROVE/TROVE_KerryCH_ca1890_MrMontaguesGreenHillsStation-photo_download_nla.obj-141608194-1.jpg


It is possible for all elements of the MetaSource to be the same entity.

A good example is The New South Wales Registry of Births Deaths & Marriages (or NSW-BDM-R0001 below) because it is the repository, the source and the authority all rolled into one.

~/media&files/NSW-BDM-R0001 /NSW-BDM-R0001-1909-02-14-WhitnallJim-I003-Birth-extract-scan.jpg

~/media&files/NSW-BDM-R0001/1909-02-14-WhitnallJim-I003-Birth-extract-scan.jpg     -------(I'll probably use this one)

Remember elements of this structure can be used for either folder or filenames or both, to indicated relationships clearly, and depending on requirements, to reduce pathway and filename length.

And the last example is an image of a church in an ebook at  http://gutenberg.net.au/ebooks06/0600151h.html

~/media&files/GutenbergNetAuR0015/Gutenberg.Net.Au-R0015_SydneyIn1948_FowlesJoseph_1948_StPhilipsChurchHill_engraving-Plate 10B_download-2017-02_0600151h_html.jpg

You can find the real books here http://trove.nla.gov.au/work/9403137 


 ---I don't think I'll put in the page plate reference in my pathway/filename at this stage. It about 160 CHAR long, I think Windoze machines can handle a pathway& filename total length of about 260 CHAR.


Anyway---------

This is the general schema. I call my familiar archives fonds now in GRAMPS as repositories.


I'll put pesky detail in the next email.


meika

http://meika.loofs-samorzewski.com

I am struggling to reason why I want my folder structure to show Repositories/sources/ when I am interested in people in the family and how they link to each other. I am happy that I record in gramps the repositories and sources for each item, but these are items of information about where the item came from or is stored, not the content of the item. If I want a picture or form pertaining to somebody, I don't want to have to start wondering in which repository folder it is stored.

Peter M.



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Folder Structure & File Naming after reading up on the French way of Archiving PART 1

paul womack
Peter Merchant wrote:

>>
> I am struggling to reason why I want my folder structure to show Repositories/sources/ when I am interested in people in the family and how they link to each other. I am happy that I record in gramps the repositories and sources for each item, but these are items of information about where the item came from or is stored, not the content of the item. If I want a picture or form pertaining to somebody, I don't want to have to start wondering in which repository folder it is stored.

Since I'm using Gramps to store my database, and being careful to use Citations, Sources
and repositories, I don't worry too much about the structure of my folders, paths
and filenames.

All files can be found from the person(s) they refer to by using Gramps.

I don't expect my file structure to be a "full tree".

  BugBear


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: -- SPAM --Re: Folder Structure & File Naming after reading up on the French way of Archiving PART 1

Ron Johnson
In reply to this post by meikamona
On 02/28/2017 09:11 AM, Peter Merchant wrote:
[snip]
I am struggling to reason why I want my folder structure to show Repositories/sources/ when I am interested in people in the family and how they link to each other. I am happy that I record in gramps the repositories and sources for each item, but these are items of information about where the item came from or is stored, not the content of the item. If I want a picture or form pertaining to somebody, I don't want to have to start wondering in which repository folder it is stored.

This is similar to what Philip Weiss wrote in http://gramps.1791082.n4.nabble.com/Folder-Structure-File-Naming-tp4679134p4679136.html

-- 
World Peace Through Nuclear Pacification

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Folder Structure & File Naming after reading up on the French way of Archiving PART 1

Rich Lakey
In reply to this post by paul womack
I have stuck with the Gallery with sub-folders.
Audio (audio recording of event)
Biography
Birth Certificate
Cemeteries
Census
Death Records
Directories
Documents
Grave Markers
Histories
Maps
Marriage Documents
Military Records
News Paper Clippings
Obituary
Photos
Ships List
Wills

None of these have anything to do with where they came from or who I got them from.  But my Person/Gallery contains them all pertaining to a person so I have one place I can see everything for a person. And the citation that tells page number etc. also has them in its gallery.  Item naming convention varies a little for the topic, but generally starts with a date. Grave Marker has death date in the format 20170228 
I've pretty much finished up documenting my census records and am working on Find A Grave records now. As they are a note, the note has type Burial. And I am confirming each on Find A Grave as I go. In the Citation has the note and in the citation/ Gallery the Grave Stone image.
When I get to obituaries I am wondering how to do them as some are Newspaper Clippings. Do they go in Obituaries or News Paper Clippings? Probably Obituary. But I can still tell they were from a News Paper as following the date published is "NP" to identify from a News Paper.
Many items in the Gallery that are an image of a document also has a note with the information transcribed. The note is attached to the citation along with the image.
And if this doesn't work I will change it.
Rich




On 02/28/2017 09:16 AM, paul womack wrote:
Peter Merchant wrote:


        
I am struggling to reason why I want my folder structure to show Repositories/sources/ when I am interested in people in the family and how they link to each other. I am happy that I record in gramps the repositories and sources for each item, but these are items of information about where the item came from or is stored, not the content of the item. If I want a picture or form pertaining to somebody, I don't want to have to start wondering in which repository folder it is stored.
Since I'm using Gramps to store my database, and being careful to use Citations, Sources
and repositories, I don't worry too much about the structure of my folders, paths
and filenames.

All files can be found from the person(s) they refer to by using Gramps.

I don't expect my file structure to be a "full tree".

  BugBear


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org



--
Have you backed up your files today?

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Folder Structure & File Naming after reading up on the French way of Archiving PART 1

Oldest1
In reply to this post by Peter Merchant

As I have not followed this thread to closely and just came across it, I may be repeating something already said.

If so, my apologies.

In creating any directory structures, one needs to be mindful of are the various limits imposed by file systems.

Depending on your working OS, you may end up with a overall path & file name length which, while supported by your working OS, may exceed allowed file name or path length or possibly directory depth for backup media if you intend to make a bare backup - i.e not zipped or compressed in any way.

For instance, some of the CD/DVD limits (for the UDF format) are:

Max Filename length: 155 bytes => 127 (wide or UTF) chars,

Max Path length:m 1023 bytes = 511 (wide or UTF) chars.

This, in my experience, applies mainly to direct uncompressed CD/DVD backups or direct backups (copies) to another branch in an OS directory, where the additional root path length of the branch may well cause the overall path length to exceed to overall limit for the OS.

The obvious work-around is to us some compression format which will preserve the original path names and directory structures.

Arnold


On 2/28/2017 7:16 AM, paul womack wrote:
Peter Merchant wrote:


        
I am struggling to reason why I want my folder structure to show Repositories/sources/ when I am interested in people in the family and how they link to each other. I am happy that I record in gramps the repositories and sources for each item, but these are items of information about where the item came from or is stored, not the content of the item. If I want a picture or form pertaining to somebody, I don't want to have to start wondering in which repository folder it is stored.
Since I'm using Gramps to store my database, and being careful to use Citations, Sources
and repositories, I don't worry too much about the structure of my folders, paths
and filenames.

All files can be found from the person(s) they refer to by using Gramps.

I don't expect my file structure to be a "full tree".

  BugBear


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: -- SPAM --Re: Folder Structure & File Naming after reading up on the French way of Archiving PART 1

Philip Weiss
In reply to this post by Ron Johnson
On 02/28/2017 09:11 AM, Peter Merchant wrote:
[snip]
I am struggling to reason why I want my folder structure to show Repositories/sources/ when I am interested in people in the family and how they link to each other. I am happy that I record in gramps the repositories and sources for each item, but these are items of information about where the item came from or is stored, not the content of the item. If I want a picture or form pertaining to somebody, I don't want to have to start wondering in which repository folder it is stored.

 
I used to have my files organized by surname and person.  What I ran into was that so much of my sources apply to multiple people:
  • census records - often have every member of a family listed, and I often have multiple families on a page in my tree
  • obituaries - often mention information about dozens of people of interest, where they live, and relationship to the decedent
  • marriage certificates - do these go under bride or groom?  And what name for the bride in countries where they take the husband's name? I don't always know the birth name.
  • birth records - lists parents as well as the person being born. sometimes lists information on additional children
  • death records - lists parents and sometimes surviving spouses, informant is often a relative I'm recording information about.
  • family histories and town histories - dozens of people are listed.
  • grave markers - often have numbers of extended family members on them
  • etc.
In fact, it's very rare for a record to be only about one person.

For a while, I tried to work around this by making a rule that these were filed under the "primary person" on a record.  But deciding who the primary person was left me unsatisfied, and if I wanted to find something for a non-primary person it was a pain.  Which of the 5 people attached to the record is the primary?  I had to look in five places in my directories, or look at the path in Gramps (which is kind of a pain) to figure it out.  I also often found myself reviewing a low-quality source and wanting to replace all uses of it with higher quality sourcing (for instance, transcriptions of obituaries from GenWeb sites, or online trees, or a set of mimeographed group sheets my grandmother provided me). While I could find them with Gramps, it wasn't smooth when they were spread all over my directories.

I found it much easier to look at an item and see what citation it's attached to, then go find the directory for that source.

Mind you, I'm not saying anyone is wrong to file by person, or that filing by source is the only option.  I'm saying this *really* worked *really* well for me.  If someone, like me, finds any of those issues to be as much of a PITA as I did, please try Proctor's advice.

But if filing by name or some other system works for you, by all means, stick with it!  I don't mean this as a backhanded recommendation.  Really, and to demonstrate...

In fact, other than photographs, my actual physical files are organized by name! I have far fewer physical pieces of paper.  And I don't need to refer back to them constantly.  So the PITA parts I ran into digitally haven't been a PITA there and name filing has been just fine.

Do what works.

Phil.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Folder Structure & File Naming after reading up on the French way of Archiving PART 1

meikamona
In reply to this post by meikamona
For instance, some of the CD/DVD limits (for the UDF format) are:
Max Filename length: 155 bytes => 127 (wide or UTF) chars,
Max Path length:m 1023 bytes = 511 (wide or UTF) chars.
This, in my experience, applies mainly to direct uncompressed CD/DVD
backups or direct backups (copies) to another branch in an OS directory,
where the additional root path length of the branch may well cause the
overall path length to exceed to overall limit for the OS.
The obvious work-around is to us some compression format which will
preserve the original path names and directory structures.
Arnold

That's a really good point. The file name length here  is a little irksome but the pathway length is better than windows. I am not personally considering burning disks, but others might.

meika 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org