Greetings,

I've noticed that Synchronet's NNTP server has 'weird' issues when mirroring binary groups. Upon further investigation I found that the 'default' content-type which was being inserted by newsutil.js was causing binary attachments in the messages to appear as text.

writeln("Content-Type: text/plain; charset=IBM437");

If I comment this line out, news readers suddenly don't have a issue
figuring out there is a attachment in the message. I know this line is important as it assumes the messages are in the old IBM437 charset which is more compatible with BBS messages.

I have read through RFC 2049 (http://www.ietf.org/rfc/rfc2049.txt) and try
as might. No matter what content-type I specified, the messages did not
display correctly.

For this reason I'm trying to figure out a workaround that would work. Such
as, somehow marking the messages that newslink imports
as 'non-charsettable' or such. Unfortunately this isn't going too well and wondered if anyone had any ideas on this issue.

~Ash-Fox

Re: newsutil.js - content-type and binary usenet groups.
By: Ash-Fox to DOVE-Net.Synchronet_JavaScript on Mon Nov 19 2007 01:00 pm

Greetings,

I've noticed that Synchronet's NNTP server has 'weird' issues when mirroring binary groups. Upon further investigation I found that the 'default' content-type which was being inserted by newsutil.js was causing binary attachments in the messages to appear as text.

writeln("Content-Type: text/plain; charset=IBM437");

The line is only inserted (from newsutil.js) if the original message does not contain a specified Content-Type header field. If you're mirroring between NNTP servers, then I would expect the original messages to have this Content-Type header field.

If I comment this line out, news readers suddenly don't have a issue figuring out there is a attachment in the message. I know this line is important as it assumes the messages are in the old IBM437 charset which is more compatible with BBS messages.

I have read through RFC 2049 (http://www.ietf.org/rfc/rfc2049.txt) and try as might. No matter what content-type I specified, the messages did not display correctly.

For this reason I'm trying to figure out a workaround that would work. Such as, somehow marking the messages that newslink imports
as 'non-charsettable' or such. Unfortunately this isn't going too well and wondered if anyone had any ideas on this issue.

Can you check if the original messages (on USENET?) have the Content-Type header field or not?

digital man (xbox-live: digitlman)

Snapple "Real Fact" #39:
An ant's smell is stronger then a dog's.
Norco, CA WX: ---.-°F, --% humidity, NaN mph --- wind, --.-- inches rain/24hrs

Digital Man wrote:

writeln("Content-Type: text/plain; charset=IBM437");

The line is only inserted (from newsutil.js) if the original message does
not contain a specified Content-Type header field.

I understood that from the comments in the code :)

If you're mirroring
between NNTP servers, then I would expect the original messages to have
this Content-Type header field.

Unfortunately, that isn't case with all messages.

Can you check if the original messages (on USENET?) have the Content-Type header field or not?

The messages that I've had problems with, do not have content-type headers.

~Ash-Fox

Re: Re: newsutil.js - content-typ
By: Ash-Fox to Digital Man on Tue Nov 20 2007 04:13 am

Digital Man wrote:

writeln("Content-Type: text/plain; charset=IBM437");

The line is only inserted (from newsutil.js) if the original message does not contain a specified Content-Type header field.

I understood that from the comments in the code :)

If you're mirroring
between NNTP servers, then I would expect the original messages to have this Content-Type header field.

Unfortunately, that isn't case with all messages.

Can you check if the original messages (on USENET?) have the Content-Type header field or not?

The messages that I've had problems with, do not have content-type headers.

I'm not really sure what to suggest in that case. <shrug>

digital man (xbox-live: digitlman)

Snapple "Real Fact" #36:
A duck's quack doesn't echo.
Norco, CA WX: ---.-°F, --% humidity, NaN mph --- wind, --.-- inches rain/24hrs

Digital Man wrote:

I'm not really sure what to suggest in that case. <shrug>

Well after a bit of thinking, I came up with this code, which appears to
work well. It's a bit of hack though, and I'm not sure if this logic does enough to determine a message came from usenet.

if(content_type==undefined) {
//Check that we're not from Fidonet
if(hdr.ftn_area==undefined) {
//If we don't have a ftn, we're likely not from usenet, needs header.
if(hdr.ftn_pid==undefined) { content_type_header(); }
}
//We're from fidonet, need a header.
else { content_type_header(); }
}
}

function content_type_header()
{
/* No content-type specified, so assume IBM code-page 437 (full ex-ASCII)
*/
writeln("Content-Type: text/plain; charset=IBM437");
writeln("Content-Transfer-Encoding: 8bit");
}

Do you think I need to add any other checks?

~Ash-Fox

Re: Re: newsutil.js - content
By: Ash-Fox to Digital Man on Tue Nov 20 2007 07:14 pm

Digital Man wrote:

I'm not really sure what to suggest in that case. <shrug>

Well after a bit of thinking, I came up with this code, which appears to work well. It's a bit of hack though, and I'm not sure if this logic does enough to determine a message came from usenet.

if(content_type==undefined) {
//Check that we're not from Fidonet
if(hdr.ftn_area==undefined) {
//If we don't have a ftn, we're likely not from usen
if(hdr.ftn_pid==undefined) { content_type_header();
}
//We're from fidonet, need a header.
else { content_type_header(); }
}
}

function content_type_header()
{
/* No content-type specified, so assume IBM code-page 437 (full ex-A */
writeln("Content-Type: text/plain; charset=IBM437");
writeln("Content-Transfer-Encoding: 8bit");
}

Do you think I need to add any other checks?

The correct way to check if a message came from a network (any network) is to see if hdr.from_net_type!=NET_NONE. If you specifically want to see if the message came from the Internet (e.g. USENET), then check if hdr.from_net_type==NET_INTERNET.

digital man (xbox-live: digitlman)

Snapple "Real Fact" #72:
The average person uses 150 gallons of water per day for personal use.
Norco, CA WX: ---.-°F, --% humidity, NaN mph --- wind, --.-- inches rain/24hrs

Digital Man wrote:

The correct way to check if a message came from a network (any network) is
to see if hdr.from_net_type!=NET_NONE. If you specifically want to see if
the message came from the Internet (e.g. USENET), then check if hdr.from_net_type==NET_INTERNET.

Ah! Much easier and less prone to error.

Thanks for the help, DM.

~Ash-Fox