How to Remove Line Breaks, Duplicate Lines and Empty Lines from Text

Copy a paragraph out of a PDF and paste it somewhere else, and it arrives broken into dozens of short lines. Paste a list and half the rows are duplicated. This is one of the most common — and most annoying — text chores there is. Here is how to fix each kind of mess, and what’s really happening under the hood.

A real paste from a PDF, cleaned. We gave our text cleaner this typical mess — This sentence was / copied out of a / PDF. / two blank lines / It has broken lines. — and ticked collapse whitespace and remove empty lines. The result keeps the line structure but fixes the spacing: This sentence was on its own line, the blank lines gone. Worth knowing before you paste: with no option ticked the tool deliberately changes nothing. Each checkbox is one transformation, so you choose exactly what happens rather than getting a single opaque “clean” button that also does things you did not want.

Input and output, verbatim. A typical PDF paste through our text cleaner with collapse whitespace and remove empty lines ticked:

This  sentence   was
copied out of a
PDF.


It has  broken lines.

becomes

This sentence was
copied out of a
PDF.
It has broken lines.

The doubled spaces and the blank lines are gone; the line structure survives. And with no box ticked the tool deliberately changes nothing — each checkbox is exactly one transformation, so you never get side effects you did not ask for.

Removing line breaks (reflowing a paragraph)

PDFs and emails often store a paragraph as many separate lines, each ending in a hard line break. When you paste it into a normal editor, it stays broken. To fix it you join the lines back together:

• Join with a space — the usual choice: line one⏎line two becomes line one line two. Use this for prose.
• Join with nothing — for data that was split mid-token and should run straight together.

The trick is knowing that a “line break” is a real character (or two: carriage return + line feed). Removing it is just replacing those characters.

There is a measurable reason to reflow before you compare or commit anything. We gave our text diff the same three words twice — once as one line, once as three — and changed nothing else:

−
one two three
+
one
+
two
+
three

Every single line came back marked as changed. To any line-based tool, line breaks are content, which is why a paragraph copied out of a PDF with hard breaks will never match its reflowed twin in a diff, a duplicate check, or a database comparison.

Removing duplicate lines

For lists — email addresses, keywords, URLs — duplicates creep in fast. Removing them means keeping the first time each line appears and dropping every repeat after it, which preserves your original order. If order doesn’t matter, sort the result A→Z afterwards so identical entries would sit together anyway.

Deleting a duplicate is the cheapest possible edit for a line-based tool, and our own text diff shows why. Removing one line out of three returns

alpha
−
beta
 
gamma

while adding one back returns

alpha
+
beta
 
gamma

One marked line each, and the untouched lines stay unmarked — compare that with the reflow example above, where every line was marked. Deduplication is a safe operation to run before a diff; reflowing is not, unless you do it to both sides.

Removing empty lines and extra spaces

Two more classics:

• Empty lines — blank rows left between paragraphs. Dropping them tightens the text.
• Extra spaces — double spaces, or tabs turned into runs of spaces. Collapsing every run down to a single space makes everything consistent.

Measured on a real paste. We gave our text cleaner this, with collapse whitespace and remove empty lines ticked:

This  sentence   was
copied out of a
PDF.


It has  broken lines.

and it returned:

This sentence was
copied out of a
PDF.
It has broken lines.

The doubled spaces and the blank lines are gone while the line structure survives — and with no box ticked it deliberately returns the input untouched, because each checkbox is exactly one transformation.

Do it all at once, privately

Instead of running find-and-replace by hand, the text cleaner applies these in a sensible order — trim each line, collapse spaces, drop empty lines, remove duplicates, sort, then join line breaks — and shows the result live. Everything runs in your browser, so even sensitive text is never uploaded.

That is also why the operations are separate rather than one “clean” button: you can remove empty lines without collapsing spaces, or collapse spaces without touching the line count. A single button would have to guess which of those you wanted.

CR, LF and CRLF: three flavors of “new line”

That invisible break isn’t always the same character. Three exist, and they come from different operating systems:

• LF (\n, line feed) — used by Linux, macOS and Unix.
• CRLF (\r\n, carriage return + line feed) — the Windows convention.
• CR (\r) on its own — classic Mac OS before OS X, and rare today.

Most of the time you never notice. But when text moves between systems, mismatched endings cause real trouble: a file can look like every line changed in a diff, a shell script can fail with a confusing error, or the whole thing shows up as one long line — sometimes with visible ^M marks. Standardizing the line endings fixes all of it at once.

The same mechanism explains CR versus LF. A line-based tool compares the bytes that end each line, so a file switching newline style looks completely rewritten even though no visible character moved — exactly the pattern above, where rewrapping alone marked every line:

−
one two three
+
one
+
two
+
three

That is why “normalise line endings” is a separate commit from any real change, and why reviewers ask for it that way.

Soft wrap is not a line break

A common mix-up: when an editor wraps a long line to fit the window, that’s a soft wrap — purely visual, with no character behind it. A hard break is a real character you can delete. Removing line breaks only touches the hard ones; soft wrapping re-flows on its own and leaves nothing to remove.

When this comes up

• Preparing a list — emails, keywords, URLs — with exactly one item per line for import.
• Pasting a paragraph into a form or chat that would otherwise arrive in fragments.
• Cleaning an email signature or a pasted quote before reusing it.

Mismatched endings are also why two near-identical files can look completely different side by side — the guide to comparing two texts shows how to see past the noise to the real changes.

Related text tools

• Counting words or characters afterwards? Use the word counter.
• Need to fix capitalization too? The case converter handles UPPER, lower, Title Case and more.