这是一个创建于 1799 天前的主题，其中的信息可能已经有所发展或是发生改变。
我尝试了很多种编码，都不能正常显示，联系上下文，我猜这可能是 emoji 之类的东西
有什么好办法可以处理这样的乱码吗？如果我要删掉它，我也得先定义它...有一个简单的思路是判断一下是不是字符是不是 ascii 里的，如果不是就直接删掉，还有更好的办法吗?
 |
1
mayx 2019-12-30 11:28:43 +08:00 via Android
正则表达式吧
|
 |
2
lululau 2019-12-30 11:34:37 +08:00
文本发上来
|
 |
3
lqs 2019-12-30 11:36:43 +08:00
猜测是用 emoji 编码成 utf8 然后用 iso-8859-1 解码了，可以把乱码发上来看看
|
 |
4
chairuosen 2019-12-30 11:52:40 +08:00
爬的 twitter 吧？叹号后面肯定是表情啦
|
 |
5
RicardoY OP |
|
7
lqs 2019-12-30 13:05:49 +08:00  2
@RicardoY
和猜测的一样 $ head train_E6oV3lV.csv |iconv -f utf8 -t iso-8859-1 id,label,tweet 1,0, @user when a father is dysfunctional and is so selfish he drags his kids into his dysfunction. #run 2,0,@user @user thanks for #lyft credit i can't use cause they don't offer wheelchair vans in pdx. #disapointed #getthanked 3,0, bihday your majesty 4,0,#model i love u take with u all the time in ur📱!!! 😙😎👄👅💦💦💦 >>> print(open('train_E6oV3lV.csv').read(1000).decode('utf8').encode('iso-8859-1').decode('utf8')) id,label,tweet 1,0, @user when a father is dysfunctional and is so selfish he drags his kids into his dysfunction. #run 2,0,@user @user thanks for #lyft credit i can't use cause they don't offer wheelchair vans in pdx. #disapointed #getthanked 3,0, bihday your majesty 4,0,#model i love u take with u all the time in ur📱!!! 😙😎👄👅💦💦💦 5,0, factsguide: society now #motivation 6,0,[2/2] huge fan fare and big talking before they leave. chaos and pay disputes when they get there. #allshowandnogo 7,0, @user camping tomorrow @user @user @user @user @user @user @user danny… 8,0,the next school year is the year for exams.😯 can't think about that 😭 #school #exams #hate #imagine #actorslife #revolutionschool #girl 9,0,we won!!! love the land!!! #allin #cavs #champions #cleveland #clevelandcavaliers … 10,0, @user @user welcome here ! i'm it's so #gr8 ! 11,0, ↝ #ireland consumer price index (mom) |
Select Language