cocoon solr No pregenerated HTML

NOTE: Do not update the repository from Cocoon files! There is a specific set of files which should be publically available and they have been carefully curated in the data repository.

Update Development

Log in and pull to development

Step 1: Log in

SSH into the development server and navigate to the data repository. [More info: if this is your first time logging in, you will need to set up server access.]

ssh username@whitman-dev.unl.edu


cd /var/local/www/data/collections/whitman-notebooks

Step 2: Check for changes

Check to see which branch is currently checked out and see if there are any unexpected changes.

git status

Typically, you should be on the dev or main branch. [More info: What to do if it is on a different branch.]

If you're on your desired branch, but there are outstanding changes, you will need to deal with them before you pull. [More info: There are files changed on the server.]

Step 3: Pull!

Once you are on the correct branch, there are no outstanding changes, and everything is copacetic, then you can pull updates.

git pull origin [current branch name]

You should either get a message saying "Already up to date" or you should see a bunch of new files show up with no errors. [More info: What if there is a problem when I git pull?]

Now you should be ready to begin updating the development website's contents!

Update the development site

Step 1. Clear Solr (optional)

If you have removed any files or changed identifiers, you will need to clear the old, no longer used file from the Solr index. If not, skip to Step 2.

You may either clear either one specific file or ALL the files. Keep in mind that if you clear all the files, the site will have no content until you repopulate the search.

WARNING: This big project uses one search index for ALL of its many parts, so you must ALWAYS specify which section you are clearing to avoid clearing the entire site.

solr_clear_index -e development -f subCategory -r notebooks

You can also clear a specific file if you only need to drop one item from the index, but know that if you use an id like `10` you may be clearing more items than you intend, so be specific if possible!

# clear one file from the index with -r [id]
#   (do not include extension)
#   (be specific with id to avoid accidentally removing more files)
solr_clear_index -e development -r wwa\.0001

[More info: other options for clearing Solr by subcategory, etc]

Step 2. Populate the search!

This is the part where you get to add your new and updated files into your site's search!

post -e development -x solr

This may take some time to run, so if you are impatient, you may want to consider only posting specific files unless if you cleared the entire index in the last step. [More info: learn about posting by file type, date, and file name.]

Check the site

This site is built in cocoon, which means that you will need to do one more thing before you can see your changes!

You are going to need to BREAK THE SITE.

This sounds scary but I promise you that it almost always, usually, well, probably is not. We need to break the site so that Cocoon will refresh its contents. We do this by introducing an intentional error in some XSLT.

cd /var/local/www/cocoon/[ your_project_dir ]

mv stylesheets/xslt/tei.p5.xsl stylesheets/xslt/tei.p5.tmp.xsl

Now, go to https://whitman-dev.unl.edu/manuscripts/notebooks/index.html and hard refresh the page. It should be broken! That's a good thing. Now let's unbreak it.

mv stylesheets/xslt/tei.p5.tmp.xsl stylesheets/xslt/tei.p5.xsl

# move back to the data directory
cd /var/local/www/data/collections/

Go back to https://whitman-dev.unl.edu/manuscripts/notebooks/index.html and refresh again. Hopefully it is unbroken and your changes are now available! [More info: what if it's still broken?]

Push changes (if needed)

Part 1: Check changes

After you've run your script, check if there are any files that have changed which you should commit.

git status

In general, files generated for the development environment will not need to be committed, so do not be surprised if there isn't anything here!

Part 2: Commit and push

If there are expected changes, such as to output/development/html, or for project-specific features, you will need to commit them.

# you may run git add repeatedly to add different directories and files
git add [file path] [or directory path]

# check it over to make sure you're adding everything you need
git status

git commit -m "message about your changes"

git push origin [current branch name]

Update Production

Log in and pull to production

Step 1: Log in

SSH into the production server and navigate to the data repository. [More info: if this is your first time logging in, you will need to set up server access.]

ssh username@whitman-prod.unl.edu


cd /var/local/www/data/collections/whitman-notebooks

Step 2: Check for changes

Check to see which branch is currently checked out and see if there are any unexpected changes.

git status

Typically, you should be on the dev or main branch. [More info: What to do if it is on a different branch.]

If you're on your desired branch, but there are outstanding changes, you will need to deal with them before you pull. [More info: There are files changed on the server.]

Step 3: Pull!

Once you are on the correct branch, there are no outstanding changes, and everything is copacetic, then you can pull updates.

git pull origin [current branch name]

You should either get a message saying "Already up to date" or you should see a bunch of new files show up with no errors. [More info: What if there is a problem when I git pull?]

Now you should be ready to begin updating the production website's contents!

Update the production site

Step 1. Clear Solr (optional)

If you have removed any files or changed identifiers, you will need to clear the old, no longer used file from the Solr index. If not, skip to Step 2.

You may either clear either one specific file or ALL the files. Keep in mind that if you clear all the files, the site will have no content until you repopulate the search.

WARNING: This big project uses one search index for ALL of its many parts, so you must ALWAYS specify which section you are clearing to avoid clearing the entire site.

solr_clear_index -e production -f subCategory -r notebooks

You can also clear a specific file if you only need to drop one item from the index, but know that if you use an id like `10` you may be clearing more items than you intend, so be specific if possible!

# clear one file from the index with -r [id]
#   (do not include extension)
#   (be specific with id to avoid accidentally removing more files)
solr_clear_index -e production -r wwa\.0001

[More info: other options for clearing Solr by subcategory, etc]

Step 2. Populate the search!

This is the part where you get to add your new and updated files into your site's search!

post -e production -x solr

This may take some time to run, so if you are impatient, you may want to consider only posting specific files unless if you cleared the entire index in the last step. [More info: learn about posting by file type, date, and file name.]

Check the site

This site is built in cocoon, which means that you will need to do one more thing before you can see your changes!

You are going to need to BREAK THE SITE.

This sounds scary but I promise you that it almost always, usually, well, probably is not. We need to break the site so that Cocoon will refresh its contents. We do this by introducing an intentional error in some XSLT.

cd /var/local/www/cocoon/[ your_project_dir ]

mv stylesheets/xslt/tei.p5.xsl stylesheets/xslt/tei.p5.tmp.xsl

Now, go to https://whitmanarchive.org/manuscripts/notebooks/index.html and hard refresh the page. It should be broken! That's a good thing. Now let's unbreak it.

mv stylesheets/xslt/tei.p5.tmp.xsl stylesheets/xslt/tei.p5.xsl

# move back to the data directory
cd /var/local/www/data/collections/

Go back to https://whitmanarchive.org/manuscripts/notebooks/index.html and refresh again. Hopefully it is unbroken and your changes are now available! [More info: what if it's still broken?]

Push changes (if needed)

Part 1: Check changes

After you've run your script, check if there are any files that have changed which you should commit.

git status

Part 2: Commit and push

If there are expected changes, such as to output/production/html, or for project-specific features, you will need to commit them.

# you may run git add repeatedly to add different directories and files
git add [file path] [or directory path]

# check it over to make sure you're adding everything you need
git status

git commit -m "message about your changes"

git push origin [current branch name]

More Information

Login and Pull

How to set up SSH

TODO

What to do if the git branch is not dev or main

TODO

There are files changed on the server

TODO

What if there is a problem when I git pull?

TODO cover permissions, merge conflict

Update the site

Clearing Solr by subcategory and more

TODO

Clearing Elasticsearch by subcategory and more

TODO possibly combine with above

Posting to Solr by specific file type, name, and updated date

TODO

HTML generation errors

TODO

Elasticsearch population errors

TODO

Check the site

Cocoon is broken

TODO

How to hard refresh

TODO

Push changes