Dealing with WRITE FPDMA QUEUED hard disk trouble

Recently, on one of my servers, I had the main hard disk being set to failsafe read-only with the following ATA errors logged in /var/log/kern.log:

ata1.00: failed command: WRITE FPDMA QUEUED
ata1.00: cmd 61/00:90:40:7e:85/04:00:ea:00:00/40 tag 18 ncq 524288 out
res 40/00:0c:40:b6:85/00:00: ea:00:00/40 Emask 0x10 (ATA bus error)
ata1.00: status: { DRDY }
ata1.00: failed command: WRITE FPDMA QUEUED
ata1.00: cmd 61/00:98:40:82:85/04:00:ea:00:00/40 tag 19 ncq 524288 out
res 40/00:0c:40:b6:85/00:00:ea:00:00/40 Emask 0x10 (ATA bus error)
ata1.00: status: { DRDY }
ata1.00: failed command: WRITE FPDMA QUEUED

It looks like a physical issue with the drive. Though there was no specific error reported by  smartctl -a /dev/sdX, and the disk was quite new (just one year old), a Western Digital Red (SATA 3 model WDC WD40EFRX-68WT0N0). The mainboard being only SATA 2, the drive is not at all pushed to its limits.

The SATA cables were the same age and SATA 3 and not in bad apparent condition except it looks like they did not stayed perfectly plugged in over time.

I switched the apparently faulty disk to the SATA connector used by the secondary hard disk and made sure they are both  properly plugged: so far, it fixed it. It really looks like a SATA cable issue.

 

Creating primitive subtitle (SubRip/.srt) file from scratch

I had a video I wanted to add a few subtitles lines here and there, without much effort. I ran:

$ apt-cache search subtitle editor
aegisub - Advanced subtitle editor
aegisub-l10n - Aegisub language packages
gaupol - subtitle editor for text-based subtitle files
libsubtitleeditor-dev - subtitleeditor lib - development files
libsubtitleeditor0 - subtitleeditor lib - runtime files
subtitleeditor - Graphical subtitle editor with sound waves representation
python-aeidon - reading, writing and manipulating text-based subtitle files
gnome-subtitles - Subtitle editor for the GNOME Desktop environment

and installed a few of them.

I was quite a disappointing experience. These softwares are more or less working but none was easy and fast enough to use for me not to prefer just using mpv and emacs, viewing with mpv and typing relevant lines in a text file with  timestamps.

Some of these softwares looked almost ok but adding new lines that was generally inconvenient (not able to insert at the exact place the playthrough is paused, or supposed to be able to but simple not working).

So, yeah, I got the simple way and built a file like :

1:01:01 adjkajdalm kadmùlajpod jaaaaaaaa aaaaaaaa aaaaaaaaaaaaaaaaaaa aaaaaaaaaaaa
11:01:01 adjkajdalm kadmùlajp odjaaaaaaaaa aaaaaaaaaaaaaaaaaa
111:01:01 adjkajda

And wrote a fast primitive script as follows:

#!/usr/bin/perl
use strict;
my $file = "THIS";
open(IN, "< $file.txt"); # input
open(OUT, "> $file.srt");
my $count = 0;
while (<IN>) {
 next if /^\s$/; #skip empty
 $count++;
 print OUT "$count\n";
 if (/^(\d{1,3})\:(\d{2}):(\d{2})\s/) { # extract time HH:MM:SS
 my ($h, $m, $s) = ($1, $2, $3);
 s/^$h:$m:$s\s//g; # remove the time from the line
 my $duration = 3; # 3 sec average duration 
 $duration = 5 if length($_) > 34; # 5 for almost two lines
 $duration = 2 if length($_) < 15; # 2 when very short
 
 my ($hplus, $mplus) = ($h, $m);
 my $splus = ($s + $duration);
 if ($splus > 60) { $mplus = ($m + 1); $splus = ($splus - 60); }
 if ($mplus > 60) { $hplus = ($h + 1); $mplus = ($mplus - 60); }
 
 print OUT sprintf("%02d:%02d:%02d,000 --> %02d:%02d:%02d,000",
 $h, $m, $s,
 $hplus, $mplus, $splus)."\n";
 } else {
 print "$count TIME ?\n"; 
 }
 print OUT $_;
 print OUT "\n";
}
close(IN);
close(OUT);
#EOF

It requires $file to be set accordingly to the name given to the input text file, obviously.

When satisfied with the written .srt, it’s convenient to embed it in the end file:

ffmpeg -i infile.mp4 -f srt -i infile.srt -c:v copy -c:a copy -c:s mov_text outfile.mp4

 

Creating an mp4 with black screen and silent audio (with ffmpeg)

Assuming you have a  black PNG image named blck.png, you can make it a mp4 video:

ffmpeg -loop 1 -i blck.png -t 900 -r 1 -c:v libx264 blck.mp4

And if you want to concatenate it with other mp4 (see previous post), it also needs to include a sound track. So, assuming your main files are in 48000hz (otherwise adjust), you can add it with the command:

ffmpeg -f lavfi -i anullsrc=r=48000 -i blck.mp4 -to 00:15:00 -c:v copy -c:a aac -strict experimental blck-snd.mp4

(note : -to needs to be set to the expected time, option -shortest fails)

 

 

Concatenating mp4 videos (with ffmpeg)

mp4 cannot be concatenated directly. But fmpeg (and so avconv I guess) can by the way of intermediary mpeg transport streams.

Here’s a dirty bash script that does that assuming you want to concat files named like MYFILE-1.mp4 MYFILE-2.mp4. Then you would just execute the script with MYFILE- as argument.

#!/bin/bash
echo $1
[ ! -e "$1"1.*4 ] && echo "nothing for $1""1.mp4" && exit

list=""
for i in `seq 1 20`; do
[ ! -e "$1"$i.*4 ] && continue
echo "$1$i.mp4 exists, creating $1$i.ts"
ffmpeg -i "$1"$i.*4 -c copy -bsf:v h264_mp4toannexb -f mpegts "$1"$i.ts
list="$list|$1"$i".ts"
done

echo "##########################################"
list=`echo "$list" | cut -c 2-`
echo "concat of $list"
ffmpeg -i "concat:$list" -c copy -bsf:a aac_adtstoasc "$1""F.mp4"
rm -f "$1"*.ts

# let user manually remove source files after checks
# EOF

How to properly set contrast on Iiyama ProLite GE2488HS-B1

In the spirit of this Finnish IT retailer listing most returned products, for once, this is not really an howto. The easiest solution I found was to return it. The contrast and luminosity is impossible to set. It changes over reboot, it changes over operating system. And even set to get high contrast, black is never black, despite Iiyama claiming that “it is also possible to adjust the brightness and the dark shades with the Black Tuner, giving greater viewing performance in shadowed areas”. I replaced it with a SyncMaster S24D340H and had none of these problems.

Side note: I also had a defective pixel, but that could happen I assumed. Well, turns out that out of 16 comments of the shop, 4 report 0 day defective pixels.

Getting back suspend/resume with KDE 5’s powerdevil and no systemd by installing an obsolete version of upower

Using a package out of date since more 2 years ago does not sound like a success story but that is the only way so far I found to get suspend/resume to work without systemd within KDE5 without headaches. pm-suspend and pm-hibernate, on the command line, work perfectly though.

Why? Because Powerdevil, KDE’s power management tool, use upower which itself deprecated pm-utils support in favor of systemd.  So, no matter whether your hardware can actually suspend and hibernate, no matter if the kernel, GNU/Linux itself, can handle, upower won’t.

When calling upower -d, it should output something with can-suspend and can-hibernate. Since they dropped support for pm-utils, it won’t if you don’t use systemd . It’ll behave as if it knew what it was doing except it does not.

I filled a bug report and this one was discarded very fast. Martin Gräßlin immediately marked it as RESOLVED DOWNSTREAM with the comment “This works fine on Debian testing. Please get in contact with your distribution to figure out why this broke in your Debian fork. You might consider of course to install systemd”. You get the idea.  Thanks to Michael Palimaka, I got confirmation that it was tied to upower version (which I guessed beforehand because of several related messages by some Ubuntu or else users – hence the mention “with upower 0.99.3 and Devuan” in my report title) and he listed working solutions: using systemd; using ConsoleKit2; using upower <=0.9.23.

Using systemd to fix a problem caused by an attempt not to use systemd? Not an option. Using ConsoleKit2? Except I have no knowledge of ConsoleKit2 being packaged yet, neither do I know which release of upower actually got ConsoleKit2 support.

So I went for the third option, the lamest obviously, that is installing obsolete, unsupported software, and put in on hold. It can be done as follow:

echo "deb http://ftp.debian.org/debian wheezy main" > /etc/apt/sources.list.d/oldstable.list
apt-get update
apt-get -t wheezy install libgnutls26 libgcrypt11 libtasn1-3 libusbmuxd1 libimobiledevice2 upower libupower-glib libplist1
echo "upower hold" | dpkg --set-selections

Then a call to upower -d gives:

Daemon:
 daemon-version: 0.9.17
 can-suspend: no
 can-hibernate no
 on-battery: no
 on-low-battery: no
 lid-is-closed: no
 lid-is-present: no
 is-docked: no

It is better but still no good. As root it’ll work, though. You need to add some PolicyKit rule to allow regular users to use it. The following assumes that powerdev group exists and that your regular users are in this group (if they are not, add them with adduser thisuser powerdev):

echo "[Suspend power group override]
Identity=unix-group:powerdev
Action=org.freedesktop.upower.suspend
ResultAny=yes
ResultInactive=yes
ResultActive=yes

[Hibernate power group override]
Identity=unix-group:powerdev
Action=org.freedesktop.upower.hibernate
ResultAny=yes
ResultInactive=yes
ResultActive=yes" > /etc/polkit-1/localauthority/50-local.d/power-group.suspend-override.pkla

Now, if done properly, upower -d returns:

Daemon:
 daemon-version: 0.9.17
 can-suspend: yes
 can-hibernate yes
 on-battery: no
 on-low-battery: no
 lid-is-closed: no
 lid-is-present: no
 is-docked: no

Logout and login should be enough to have back suspend and hibernate within KDE5.

So this works. For now. But there is no doubt, this is wrong in so many ways. I wonder for how long it will be possible to run a modern desktop environment on GNU/Linux, and not on systemd/whatever.