czwartek, 28 kwietnia 2016

GitLab zepsuł dodawanie w wiki pages podczas aktualizacji

Zainstalowałem sobie GitLaba na prośbę naszych programistów, i tak sobie przez pewien czas stał i się kurzył. Aż do czasu jak stwierdziliśmy, że sobie wiki na to przeniesiemy bo ładnie widać kto, co, kiedy zmienił/wpisał, itp. Przy okazji odkurzania na Debianie poszło aptitude upgrade, żeby wszystko ładnie działało. Niestaty trafiło na aktualizacje GitLab`a. Na pierwszy rzut oka działało bardzo ładnie, aż do pierwszej próby zaktualizowania wiki. 500 Error :/. Zaglądamy do logów i widzimy:

/var/log/gitlab/gitlab-rails/production.log

NoMethodError (undefined method `username' for #<Group:0x00000007e195b8>):
  app/models/project_wiki.rb:158:in `default_message'
  app/models/project_wiki.rb:152:in `commit_details'
  app/models/project_wiki.rb:91:in `create_page'
  app/models/wiki_page.rb:183:in `save'
  app/models/wiki_page.rb:143:in `create'
  app/services/wiki_pages/create_service.rb:6:in `execute'
  app/controllers/projects/wikis_controller.rb:58:in `create'
  lib/gitlab/middleware/go.rb:16:in `call'

Okazało się, że najnowsza wersja tak ma i już ..... Po godzinie czesania google udało się znaleźć rozwiązanie problemu przez społeczność.

W pierwszej kolejności sprawdzamy, czy z samym Gitlabem jest wsio ok

Sprawdzamy czy bazy wstały prawidłowo:

gitlab-rake db:migrate:status

database: gitlabhq_production

 Status   Migration ID    Migration Name
--------------------------------------------------
   up     20121220064453  Init schema
   up     20130102143055  Rename owner to creator for project
   up     20130110172407  Add public to project
   up     20130123114545  Add issues tracker to project
   up     20130125090214  Add user permissions
   up     20130131070232  Remove private flag from project
   up     20130206084024  Add description to namsespace
   up     20130207104426  Add description to teams
   up     20130211085435  Add issues tracker id to project
   up     20130214154045  Rename state to merge status in milestone
   up     20130218140952  Add state to issue
   up     20130218141038  Add state to merge request
   up     20130218141117  Add state to milestone
   up     20130218141258  Convert closed to state in issue
   up     20130218141327  Convert closed to state in merge request
   up     20130218141344  Convert closed to state in milestone
   up     20130218141444  Remove merged from merge request
   up     20130218141507  Remove closed from issue
   up     20130218141536  Remove closed from merge request
   up     20130218141554  Remove closed from milestone
   up     20130220124204  Add new merge status to merge request
   up     20130220125544  Convert merge status in merge request
   up     20130220125545  Remove merge status from merge request
   up     20130220133245  Rename new merge status to merge status in milestone
   up     20130304104623  Add state to user
   up     20130304104740  Convert blocked to state
   up     20130304105317  Remove blocked from user
   up     20130315124931  User color scheme
   up     20130318212250  Add snippets to features
   up     20130319214458  Create forked project links
   up     20130323174317  Add private to snippets
   up     20130324151736  Add type to snippets
   up     20130324172327  Change project id to null in snipepts
   up     20130324203535  Add type value for snippets
   up     20130325173941  Add notification level to user
   up     20130326142630  Add index to users authentication token
   up     20130403003950  Add last activity column into project
   up     20130404164628  Add notification level to user project
   up     20130410175022  Remove wiki table
   up     20130419190306  Allow merges for forks
   up     20130506085413  Add type to key
   up     20130506090604  Create deploy keys projects
   up     20130506095501  Remove project id from key
   up     20130522141856  Add more fields to service
   up     20130528184641  Add system to notes
   up     20130611210815  Increase snippet text column size
   up     20130613165816  Add password expires at to users
   up     20130613173246  Add created by id to user
   up     20130614132337  Add improted to project
   up     20130617095603  Create users groups
   up     20130621195223  Add notification level to user group
   up     20130622115340  Add more db index
   up     20130624162710  Add fingerprint to key
   up     20130711063759  Create project group links
   up     20130804151314  Add st diff to note
   up     20130809124851  Add permission check to user
   up     20130812143708  Add import url to project
   up     20130819182730  Add internal ids to issues and mr
   up     20130820102832  Add access to project group link
   up     20130821090530  Remove deprecated tables
   up     20130821090531  Add internal ids to milestones
   up     20130909132950  Add description to merge request
   up     20130926081215  Change owner id for group
   up     20131005191208  Add avatar to users
   up     20131009115346  Add confirmable to users
   up     20131106151520  Remove default branch
   up     20131112114325  Create broadcast messages
   up     20131112220935  Add visibility level to projects
   up     20131129154016  Add archived to projects
   up     20131130165425  Add color and font to broadcast messages
   up     20131202192556  Add event fields for web hook
   up     20131214224427  Add hide no ssh key to users
   up     20131217102743  Add recipients to service
   up     20140116231608  Add website url to users
   up     20140122112253  Create merge request diffs
   up     20140122114406  Migrate mr diffs
   up     20140122122549  Remove m rdiff fields
   up     20140125162722  Add avatar to projects
   up     20140127170938  Add group avatars
   up     20140209025651  Create emails
   up     20140214102325  Add api key to services
   up     20140304005354  Add index merge request diffs on merge request
   up     20140305193308  Add tag push hooks to project hook
   up     20140312145357  Add import status to project
   up     20140313092127  Migrate already imported projects
   up     20140407135544  Fix namespaces
   up     20140414131055  Change state to allow empty merge request diffs
   up     20140415124820  Limits to mysql
   up     20140416074002  Add index on iid
   up     20140416185734  Index on current sign in at
   up     20140428105831  Add notes index updated at
   up     20140502115131  Add repo size to db
   up     20140502125220  Migrate repo size
   up     20140611135229  Add position to merge request
   up     20140625115202  Create users star projects
   up     20140729134820  Create labels
   up     20140729140420  Create label links
   up     20140729145339  Migrate project tags
   up     20140729152420  Migrate taggable labels
   up     20140730111702  Add index to labels
   up     20140903115954  Migrate to new shell
   up     20140907220153  Serialize service properties
   up     20140914113604  Add members table
   up     20140914145549  Migrate to new members model
   up     20140914173417  Remove old member tables
   up     20141006143943  Move slack service to webhook
   up     20141007100818  Add visibility level to snippet
   up     20141118150935  Add audit event
   up     20141121133009  Add timestamps to members
   up     20141121161704  Add identity table
   up     20141205134006  Add locked at to merge request
   up     20141216155758  Create doorkeeper tables
   up     20141217125223  Add owner to application
   up     20141223135007  Add import data to project table
   up     20141226080412  Add developers can push to protected branches
   up     20150108073740  Create application settings
   up     20150116234544  Add home page url for application settings
   up     20150116234545  Add gitlab access token to user
   up     20150125163100  Add default branch protection setting
   up     20150205211843  Add timestamps to identities
   up     20150206181414  Add index to created at
   up     20150206222854  Add notification email to user
   up     20150209222013  Add missing index
   up     20150211172122  Add template to service
   up     20150211174341  Allow null in services project
   up     20150213104043  Add twitter sharing enabled to application settings
   up     20150213114800  Add hide no password to user
   up     20150213121042  Add password automatically set to user
   up     20150217123345  Add bitbucket access token and secret to user
   up     20150219004514  Add events to services
   up     20150223022001  Set missing last activity at
   up     20150225065047  Add note events to services
   up     20150301014758  Add restricted visibility levels to application settings
   up     20150306023106  Fix namespace duplication
   up     20150306023112  Add unique index to namespace
   up     20150310194358  Add version check to application settings
   up     20150313012111  Create subscriptions table
   up     20150320234437  Add location to user
   up     20150324155957  Set incorrect assignee id to null
   up     20150327122227  Add public to key
   up     20150327150017  Add import data to project
   up     20150327223628  Add devise two factor to users
   up     20150328132231  Add max attachment size to application settings
   up     20150331183602  Add devise two factor backupable to users
   up     20150406133311  Add invite data to member
   up     20150411000035  Fix identities
   up     20150411180045  Rename buildbox service
   up     20150413192223  Add public email to users
   up     20150417121913  Create project import data
   up     20150417122318  Remove import data from project
   up     20150421120000  Remove periods at ends of usernames
   up     20150423033240  Add default project visibililty to application settings
   up     20150425164646  Gitlab change collation for tag names.acts as taggable on engine
   up     20150425164647  Remove duplicate tags
   up     20150425164648  Add missing unique indices.acts as taggable on engine
   up     20150425164649  Add taggings counter cache to tags.acts as taggable on engine
   up     20150425164650  Add missing taggable index.acts as taggable on engine
   up     20150425164651  Change collation for tag names.acts as taggable on engine
   up     20150425173433  Add default snippet visibility to app settings
   up     20150429002313  Remove abandoned group members records
   up     20150502064022  Add restricted signup domains to application settings
   up     20150509180749  Convert legacy reference notes
   up     20150516060434  Add note events to web hooks
   up     20150529111607  Add user oauth applications to application settings
   up     20150529150354  Add after sign out path for application settings
   up     20150609141121  Add session expire delay for application settings
   up     20150610065936  Add dashboard to users
   up     20150620233230  Add default otp required for login value
   up     20150713160110  Add project view to users
   up     20150717130904  Add commits count to project
   up     20150730122406  Add updated by to issuables and notes
   up     20150806104937  Create abuse reports
   up     20150812080800  Add settings import sources
   up     20150814065925  Remove oauth tokens from users
   up     20150817163600  Deduplicate user identities
   up     20150818213832  Add sent notifications
   up     20150824002011  Add enable ssl verification
   up     20150826001931  Add ci tables
   up     20150902001023  Add template to label
   up     20150914215247  Add ci tags
   up     20150915001905  Enable ssl verification by default
   up     20150916000405  Enable ssl verification for web hooks
   up     20150916114643  Add help page text to application settings
   up     20150916145038  Add index for committed at and
   up     20150918084513  Add ci enabled to application settings
   up     20150918161719  Remove invalid milestones from merge requests
   up     20150920010715  Add consumed timestep to users
   up     20150920161119  Add line code to sent notification
   up     20150924125150  Add project id to ci commit
   up     20150924125436  Migrate project id for ci commits
   up     20150930001110  Merge request error field
   up     20150930095736  Add null to name for ci projects
   up     20150930110012  Add group share lock
   up     20151002112914  Add stage idx to builds
   up     20151002121400  Add index for builds
   up     20151002122929  Add ref and tag to builds
   up     20151002122943  Migrate ref and tag to build
   up     20151005075649  Add user id to build
   up     20151005150751  Add layout option for users
   up     20151005162154  Remove ci enabled from application settings
   up     20151007120511  Namespaces projects path lower indexes
   up     20151008110232  Add users lower username email indexes
   up     20151008123042  Add type and description to builds
   up     20151008130321  Migrate name to description for builds
   up     20151008143519  Add admin notification email setting
   up     20151012173029  Set jira service api url
   up     20151013092124  Add artifacts file to builds
   up     20151016131433  Add ci projects gl project id index
   up     20151016195451  Add ci builds and projects indexes
   up     20151016195706  Add notes line code index
   up     20151019111551  Fix build tags
   up     20151019111703  Fail build without names
   up     20151020145526  Add services template index
   up     20151020173516  Ci limits to mysql
   up     20151020173906  Add ci builds index for status
   up     20151023112551  Fail build with empty name
   up     20151023144219  Remove satellites
   up     20151026182941  Add project path index
   up     20151028152939  Add merge when build succeeds to merge request
   up     20151103001141  Add public to group
   up     20151103133339  Add shared runners setting
   up     20151103134857  Create lfs objects
   up     20151103134958  Create lfs objects projects
   up     20151104105513  Add file to lfs objects
   up     20151105094515  Create releases
   up     20151106000015  Add is award to notes
   up     20151109100728  Add max artifacts size to application settings
   up     20151109134526  Add issues state index
   up     20151109134916  Add projects visibility level index
   up     20151110125604  Add import error to project
   up     20151114113410  Add index for lfs oid and size
   up     20151116144118  Add unique for lfs oid index
   up     20151118162244  Add projects public index
   up     20151201203948  Raise hook url limit
   up     20151203162133  Add hide project limit to users
   up     20151203162134  Add build events to services
   up     20151209144329  Migrate ci web hooks
   up     20151209145909  Migrate ci emails
   up     20151210030143  Add unlock token to user
   up     20151210072243  Add runners registration token to application settings
   up     20151210125232  Migrate ci slack service
   up     20151210125927  Migrate ci hip chat service
   up     20151210125928  Add ci to project
   up     20151210125929  Add project id to ci
   up     20151210125930  Migrate ci to project
   up     20151210125931  Add index to ci tables
   up     20151210125932  Drop null for ci tables
   up     20151218154042  Add tfa to application settings
   up     20151221234414  Add tfa additional fields
   up     20151224123230  Rename emojis
   up     20151228111122  Remove public from namespace
   up     20151228150906  Influxdb settings
   up     20151228175719  Add recaptcha to application settings
   up     20151229102248  Influxdb udp port setting
   up     20151229112614  Influxdb remote database setting
   up     20151230132518  Add artifacts metadata to ci build
   up     20151231152326  Add akismet to application settings
   up     20151231202530  Remove alert type from broadcast messages
   up     20160106162223  Add index milestones title
   up     20160106164438  Remove influxdb credentials
   up     20160109054846  Create spam logs
   up     20160113111034  Add metrics sample interval
   up     20160118155830  Add sentry to application settings
   up     20160118232755  Add ip blocking settings to application settings
   up     20160119111158  Add services category
   up     20160119112418  Add services default
   up     20160119145451  Add ldap email to users
   up     20160120172143  Add base commit sha to merge request diffs
   up     20160121030729  Add email author in body to application settings
   up     20160122185421  Add pending delete to project
   up     20160128212447  Remove ip blocking settings from application settings
   up     20160128233227  Change lfs objects size column
   up     20160129135155  Remove dot atom path ending of projects
   up     20160129155512  Add merge commit sha to merge requests
   up     20160202091601  Add erasable to ci build
   up     20160202164642  Add allow guest to access builds project
   up     20160204144558  Add real size to merge request diffs
   up     20160209130428  Add index to snippet
   up     20160212123307  Create tasks
   up     20160217100506  Add description to label
   up     20160217174422  Add note to tasks
   up     20160220123949  Rename tasks to todos
   up     20160222153918  Create appearances ce
   up     20160223192159  Add confidential to issues
   up     20160225090018  Add delete at to issues
   up     20160225101956  Add delete at to merge requests
   up     20160226114608  Add trigram indexes for searching
   up     20160227120001  Add event field for web hook
   up     20160227120047  Add event to services
   up     20160229193553  Add main language to repository
   up     20160301124843  Add visibility level to groups
   up     20160302151724  Add import credentials to project import data
   up     20160302152808  Remove wrong import url from projects
   up     20160305220806  Remove expires at from snippets
   up     20160307221555  Disallow blank line code on note
   up     20160308212903  Add default group visibility to application settings
   up     20160309140734  Fix todos
   up     20160310124959  Add due date to issues
   up     20160310185910  Add external flag to users
   up     20160314143402  Projects add pushes since gc
   up     20160315135439  Project add repository check
   up     20160316123110  Ci runners token index
   up     20160316192622  Change target id to null on todos
   up     20160316204731  Add commit id to todos
   up     20160317092222  Add moved to to issue
   up     20160320204112  Index namespaces on visibility level
   up     20160324020319  Remove todos for deleted issues
   up     20160328112808  Create notification settings
   up     20160328115649  Migrate new notification setting
   up     20160328121138  Add notification setting index
   up     20160329144452  Add index on pending delete projects
   up     20160331133914  Remove todos for deleted merge requests
   up     20160331223143  Remove twitter sharing enabled from application settings
   up     20160412140240  Add repository checks enabled setting
   up     20160415133440  Add shared runners text to application settings
   up     20160419120017  Add metrics packet size
   up     20160421130527  Disable repository checks

Jeśli są wszystkie up to jest ok. Jak nie trzeba je przemigrować ponownie:

gitlab-rake db:migrate

Następnie sprawdzamy poprawność całej "instancji" GitLaba:

gitlab-rake gitlab:check SANITIZE=true
Checking GitLab Shell ...

GitLab Shell version >= 2.7.2 ? ... OK (2.7.2)
Repo base directory exists? ... yes
Repo base directory is a symlink? ... no
Repo base owned by git:git? ... yes
Repo base access is drwxrws---? ... yes
hooks directories in repos are links: ...
7/1 ... repository is empty
11/3 ... repository is empty
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Check directories and files:
        /var/opt/gitlab/git-data/repositories: OK
        /var/opt/gitlab/.ssh/authorized_keys: OK
Test redis-cli executable: redis-cli 2.8.24
Send ping to redis server: PONG
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

Reply by email is disabled in config/gitlab.yml

Checking Reply by email ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
projects have namespace: ...
7/1 ... yes
11/3 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.1.8)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 2.7.3 ? ... yes (2.7.4)
Active users: 6

Checking GitLab ... Finished

Jak tu nam wywali na czerwono jakiś błąd to od razu podpowie jak go naprawić. Naprawiamy puszczamy polecenie jeszcze raz i weryfikujemy. Jeśli to nie pomoże to zabieramy się za naprawę błędu. Edytujemy plik : /opt/gitlab/embedded/service/gitlab-rails/app/models/project_wiki.rb.

linie 158:

   "#{@user.username} #{action} page: #{title}"

modyfikujemy tak:

   "#{@user.name} #{action} page: #{title}"


Następnie edytujemy plik: /opt/gitlab/embedded/service/gitlab-rails/app/services/wiki_pages/create_service.rb

module WikiPages
  class CreateService < WikiPages::BaseService
    def execute
      page = WikiPage.new(@project.wiki)

      if page.create(@params)
        execute_hooks(page, 'create')
      end

      page
    end
  end
end

Zamieniamy na:

module WikiPages
  class UpdateService < WikiPages::BaseService
    def execute(page)
      if page.update(@params[:content], @params[:format], @params[:message])
        execute_hooks(page, 'update')
      end

      page
    end
  end
end

Restartujemy GitLaba testujemy i powinno być Ok. źródło do rozwiązania (szczegóły w komentarzach są):
https://gitlab.com/gitlab-org/gitlab-ce/issues/15623#note_5054714
https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/3924/diffs#diff-2


piątek, 8 kwietnia 2016

Wyłączenie pojedynczych katalogów z zabezpieczenia hasłem w .htaccess

Tak się czasami za dzieje, że musimy witrynę zabezpieczyć hasłem całą, ale do malutkiego podfolderu musi być normalny dostęp. Przedstawię poniżej najprostszy sposób wprowadzony i przetestowany na Apache2 2.4.x. Tworzymy/modyfikujemy plik .htaccess który zabezpiecza naszą witrynę (nie będę opisywał składni pliku do zabezpieczania witryny). Jak już przy wchodzeniu na stronę wszędzie pyta się nam o hasło przechodzimy do odbezpieczania interesujących nas katalogów.
W subfolderach, które chcemy mieć wolne umieszczamy plik .htaccess o następującej składni:

Order Allow,Deny
Allow from all
Satisfy any

To powinno wystarczyć. Jeśli jednak ciągle prosi o hasło, to należy zobaczyć, czy po anulowaniu żądania hasła wyświetla nam jakiekolwiek treści. Jeśli tak to znaczy, że strona odwołuje się także do innych folderów, w których także powinniśmy umieścić powyższy plik .htaccess np folder z grafiką, lub stylem css.